Compare commits

...

51 Commits

Author SHA1 Message Date
52d636039d Merge branch 'master' of gitlab.com:LoveIsGrief/muwire into new-icon 2020-01-25 16:34:40 +01:00
483f92836c Added adapted @Shoalsteed icons 2020-01-25 16:33:49 +01:00
Zlatin Balevsky
83546d68d2 Merge pull request #37 from LoveIsGrief/change-persister
Introduce persister that uses a directory structure
2020-01-25 14:36:41 +00:00
a891c83518 Only persist downloaded files if sharing thereof is enabled
Otherwise we might inadvertently share downloads
2020-01-25 15:25:48 +01:00
aa56cc23c0 Cache base 64 path hash
Can't do it in constructor without an ugly try/catch
 therefore this is done on demand
2020-01-25 15:20:38 +01:00
a2b37ef567 Persist downloaded files 2020-01-25 15:06:12 +01:00
4bc04ae631 Revert "Reduce log levels in Connection"
This reverts commit dcd233b7
2020-01-25 15:01:21 +01:00
56da9a16b0 Set FileLoadedEvent::source in the subclass
Setting it in the super class means we don't set the right value for every case
2020-01-25 15:00:48 +01:00
2935ee1a1d Remove unnecessary executor
It was doing nothing but starting and stopping
2020-01-25 14:49:59 +01:00
855183397b Remove TODO
There's already an issue open https://github.com/zlatinb/muwire/issues/35
2020-01-22 21:35:54 +01:00
e27704c1af Make sure migration from PersisterService works
this.getClass() and this.class kept resolving to Class.
Using a string is much simpler

mkdirs() is also necessary because the directory structure doesn't exist
 when persistFile is called the first time
2020-01-22 20:59:05 +01:00
5c18b4a141 Add more logs PersisterFolderService 2020-01-22 15:12:22 +01:00
dcd233b7ad Reduce log levels in Connection
Too verbose
2020-01-22 15:12:01 +01:00
7cee8a28ba FileLoadedEvent should include class when coming from old persister
Otherwise the new PersisterFolderService won't migrate
2020-01-22 15:07:00 +01:00
7446fc949a Remove UIPersistFilesEvent
Hashing is done per file now and those are triggered by individual events
2020-01-22 13:00:55 +01:00
598ab90f63 Clear up the event path when starting up the old and new persisters
The new persister won't load anything until the old one has finished
2020-01-22 12:36:34 +01:00
043028c296 Introduce PersisterFolderService to replace PersisterService
An attempt at automatically migrate from PersisterService was made, but the events aren't triggered in the right order.
We need to make sure that we don't trigger the "AllFilesLoadedEvent" before the migration is done
2020-01-21 23:34:33 +01:00
cd1757fac3 Use Java 11
Java9 isn't available on Ubuntu anymore, which would make development harder
2020-01-19 21:46:47 +01:00
9d4b365e63 Log the time it take to persist files and hashes 2020-01-19 21:43:03 +01:00
8ac3660b36 Merge branches 'master' and 'new-icon' of gitlab.com:LoveIsGrief/muwire into new-icon 2020-01-14 23:37:51 +01:00
Zlatin Balevsky
b12d57e30a fix bracket 2020-01-14 20:27:21 +00:00
Zlatin Balevsky
f33d1b6db3 move the docker documentation to the wiki 2020-01-14 20:26:47 +00:00
Zlatin Balevsky
9e451460da change to my repo 2020-01-14 19:24:32 +00:00
Zlatin Balevsky
ffa52c129a Merge pull request #33 from LoveIsGrief/32-docker-image
Docker image
2020-01-14 19:21:35 +00:00
b779fb75a0 docker: Remove incompletes warning from README
#32 - Docker image
2020-01-14 20:11:34 +01:00
fbe6b53278 docker: Make sure build directories are ignored
#32 - Docker image
2020-01-14 19:20:11 +01:00
b2bd95788d docker: Try minimizing size using add-pkg and del-pkg
As described in https://github.com/jlesage/docker-baseimage-gui#addingremoving-packages

#32 - Docker image
2020-01-14 19:19:47 +01:00
83d4a2624b docker: Add bisentenialwrug/muwire to README
To be replaced later by @zlatinb's repo

#32 - Docker image
2020-01-14 18:47:28 +01:00
03e20e21aa Remove unnecessary quotes from properties files
There doesn't seem to be a special treatment of them
 in properties files

#32 - Docker image
2020-01-14 18:42:51 +01:00
8a08955675 Remove quotes from i2cp.tcp.port setting
For some reason it really doesn't like that and
 subsequently can't connect to the host

#32 - Docker image
2020-01-14 17:52:52 +01:00
4ec54ebe54 docker: Quote the IP-address in i2p.properties
#32 - Docker image
2020-01-14 17:36:45 +01:00
758af6f48e docker: Make sure APP_HOME is editable by the user
Otherwise MuWire won't be able to write into the home

#32 - Docker image
2020-01-14 17:14:41 +01:00
a7bdd47fcd docker: Add more files to ignore
Helps with build speed on the local machine

#32 - Docker image
2020-01-14 17:00:07 +01:00
f7caa77a18 docker: Include the MuWire icon for the webview
#32 - Docker image
2020-01-14 16:59:39 +01:00
7641f64536 docker: Add default MuWire.properties without nickname
#32 - Docker image
2020-01-14 16:59:13 +01:00
02baaace48 Merge branch 'master' of https://github.com/zlatinb/muwire into 32-docker-image 2020-01-14 16:48:12 +01:00
ef06cadde3 Icon suggestion 2020-01-14 16:30:43 +01:00
Zlatin Balevsky
d90067ff39 prompt for nickname even if MuWire.properties exists so that docker can ship a MuWire.properties #32 2020-01-14 14:17:18 +00:00
c910a215f5 Add the /incompletes docker volume
It won't be used by default though

#32 - Docker image
2020-01-14 13:07:37 +01:00
65e073b1b9 Use defaults for the i2p.properties
This will help writing custom properties
 as not everthing will have to be specified in them

#32 - Docker image
2020-01-14 12:29:05 +01:00
489a7518c3 Attempt to reduce size a bit more
- Ignore the cruft when building
 - Remove the correct temporary directory

#32 - Docker image
2020-01-14 01:09:39 +01:00
3733e48bbd Force set the port
The default isn't used in the code.
That should be fixed, but I'm too tired right now

#32 - Docker image
2020-01-14 00:29:33 +01:00
c3723a1348 Try to minimize image size
#32 - Docker image
2020-01-14 00:15:01 +01:00
0e0f52bc77 Retry: Set a home directory for the "app" user
Apparently it's done differently in the parent image,
 so we just overwrite it.

Hopefully now the app user will have a home

#32 - Docker image
2020-01-13 23:38:04 +01:00
60b9e990cf Set a home directory for the "app" user
#32 - Docker image
2020-01-13 21:34:50 +01:00
28ad0ae30f Add --name to docker run command
#32 - Docker image
2020-01-13 20:29:28 +01:00
9142de85cd Correct the link to the i2cp_config.png
#32 - Docker image
2020-01-13 19:51:20 +01:00
4eb31c11e3 Write README and cleanup inconsistencies
#32 - Docker image
2020-01-13 18:42:30 +01:00
e8afe358a5 First Dockerfile with GUI that starts
It doesn't continue yet as it seems to be waiting for a connection
 to I2P... or something else 🤷#32 - Docker image
2020-01-13 17:07:56 +01:00
Zlatin Balevsky
3db4317fc1 more items 2020-01-01 11:26:59 +00:00
Zlatin Balevsky
5ad2b28527 more items 2020-01-01 09:19:46 +00:00
22 changed files with 1032 additions and 203 deletions

12
.dockerignore Normal file
View File

@@ -0,0 +1,12 @@
# Dot directories
.gradle/
.idea/
.git/
# Build directories
build/
**/build/
# We execute COPY . .
# Modifying these files would unnecessarily invalidate the build context
Dockerfile

64
Dockerfile Normal file
View File

@@ -0,0 +1,64 @@
FROM jlesage/baseimage-gui:alpine-3.10-glibc
# Docker image version is provided via build arg.
ARG DOCKER_IMAGE_VERSION=unknown
# JDK version
ARG JDK=11
# Important directories
ARG TMP_DIR=/muwire-tmp
ENV APP_HOME=/muwire
# Define working directory.
WORKDIR $TMP_DIR
# Put sources into dir
COPY . .
# Install final dependencies
RUN add-pkg openjdk${JDK}-jre
# Build and untar in future distribution dir
RUN add-pkg --virtual openjdk${JDK}-jdk \
&& ./gradlew --no-daemon clean assemble \
&& mkdir -p ${APP_HOME} \
# Extract to ${APP_HOME and ignore the first dir
# First dir in tar is the "MuWire-<version>"
&& tar -C ${APP_HOME} --strip 1 -xvf gui/build/distributions/MuWire*.tar \
# Cleanup
&& rm -rf "${TMP_DIR}" /root/.gradle /root/.java \
&& del-pkg openjdk${JDK}-jdk
WORKDIR ${APP_HOME}
# Maximize only the main/initial window.
RUN \
sed-patch 's/<application type="normal">/<application type="normal" title="MuWire">/' \
/etc/xdg/openbox/rc.xml
# Generate and install favicons.
RUN \
APP_ICON_URL=https://github.com/zlatinb/muwire/raw/master/gui/griffon-app/resources/MuWire-128x128.png && \
install_app_icon.sh "$APP_ICON_URL"
# Add files.
COPY docker/rootfs/ /
# Set environment variables.
ENV APP_NAME="MuWire" \
S6_KILL_GRACETIME=8000
# Define mountable directories.
VOLUME ["$APP_HOME/.MuWire"]
VOLUME ["/incompletes"]
VOLUME ["/output"]
# Metadata.
LABEL \
org.label-schema.name="muwire" \
org.label-schema.description="Docker container for MuWire" \
org.label-schema.version="$DOCKER_IMAGE_VERSION" \
org.label-schema.vcs-url="https://github.com/zlatinb/muwire" \
org.label-schema.schema-version="1.0"

View File

@@ -6,7 +6,7 @@ The current stable release - 0.6.8 is avaiable for download at https://muwire.co
You can find technical documentation in the [doc] folder. Also check out the [Wiki] for various other documentation.
### Building
## Building
You need JDK 9 or newer. After installing that and setting up the appropriate paths, just type
@@ -21,7 +21,7 @@ If you want to run the unit tests, type
If you want to build binary bundles that do not depend on Java or I2P, see the [muwire-pkg] project
### Running the GUI
## Running the GUI
Type
```
@@ -32,20 +32,24 @@ If you have an I2P router running on the same machine that is all you need to do
[Default I2CP port]\: `7654`
### Running the CLI
## Running the CLI
Look inside `cli-lanterna/build/distributions`. Untar/unzip one of the `shadow` files and then run the jar contained inside by typing `java -jar cli-lanterna-x.y.z-all.jar` in a terminal. The CLI will ask you about the router host and port on startup, no need to edit any files. However, the CLI does not have an options window yet, so if you need to change any options you will need to edit the configuration files. The CLI options are documented here [cli options]
The CLI is under active development and doesn't have all the features of the GUI.
### Running the Web UI / Plugin
## Running the Web UI / Plugin
There is a Web-based UI under development. It is intended to be run as a plugin to the Java I2P router. Instructions how to build it are available at the wiki [Plugin] page.
### Translations
## Docker
MuWire is available as a Docker image. For more information see the [Docker] page.
## Translations
If you want to help translate MuWire, instructions are on the wiki https://github.com/zlatinb/muwire/wiki/Translate
### GPG Fingerprint
## GPG Fingerprint
```
471B 9FD4 5517 A5ED 101F C57D A728 3207 2D52 5E41
@@ -61,3 +65,5 @@ You can find the full key at https://keybase.io/zlatinb
[cli options]: https://github.com/zlatinb/muwire/wiki/CLI-Configuration-Options
[I2P Github]: https://github.com/i2p/i2p.i2p
[Plugin]: https://github.com/zlatinb/muwire/wiki/Plugin
[Docker]: https://github.com/zlatinb/muwire/wiki/Docker
[jlesage/docker-baseimage-gui]: https://github.com/jlesage/docker-baseimage-gui

View File

@@ -17,6 +17,9 @@ This helps with scalability
* Persist trust immediately
* Check if user-selected download and incomplete locations exist and are writeable
* Enum i18n
* Ability to share trust list only with trusted users
* Confidential files visible only to certain users
* Public Feed feature
### Chat
* echo "unknown/innappropriate command" in the console
@@ -24,6 +27,7 @@ This helps with scalability
* Style timestamps and persona names
* enforce # in room names or ignore it
* auto-create/join channel on server start
* jump from notification window to room with message
### Swing GUI
* I2P Status panel - display message when connected to external router

View File

@@ -21,7 +21,6 @@ import com.muwire.core.filecert.UICreateCertificateEvent
import com.muwire.core.files.DirectoryUnsharedEvent
import com.muwire.core.files.FileSharedEvent
import com.muwire.core.files.FileUnsharedEvent
import com.muwire.core.files.UIPersistFilesEvent
class FilesView extends BasicWindow {
private final FilesModel model
@@ -84,7 +83,6 @@ class FilesView extends BasicWindow {
Button unshareButton = new Button("Unshare", {
core.eventBus.publish(new FileUnsharedEvent(unsharedFile : sf))
core.eventBus.publish(new UIPersistFilesEvent())
MessageDialog.showMessageDialog(textGUI, "File Unshared", "Unshared "+sf.getFile().getName(), MessageDialogButton.OK)
} )
Button addCommentButton = new Button("Add Comment", {

View File

@@ -1,5 +1,8 @@
package com.muwire.core
import com.muwire.core.files.PersisterDoneEvent
import com.muwire.core.files.PersisterFolderService
import java.nio.charset.StandardCharsets
import java.util.concurrent.atomic.AtomicBoolean
@@ -31,7 +34,6 @@ import com.muwire.core.filecert.UIFetchCertificatesEvent
import com.muwire.core.filecert.UIImportCertificateEvent
import com.muwire.core.files.FileDownloadedEvent
import com.muwire.core.files.FileHashedEvent
import com.muwire.core.files.FileHashingEvent
import com.muwire.core.files.FileHasher
import com.muwire.core.files.FileLoadedEvent
import com.muwire.core.files.FileManager
@@ -41,7 +43,7 @@ import com.muwire.core.files.HasherService
import com.muwire.core.files.PersisterService
import com.muwire.core.files.SideCarFileEvent
import com.muwire.core.files.UICommentEvent
import com.muwire.core.files.UIPersistFilesEvent
import com.muwire.core.files.AllFilesLoadedEvent
import com.muwire.core.files.DirectoryUnsharedEvent
import com.muwire.core.files.DirectoryWatchedEvent
@@ -74,10 +76,8 @@ import net.i2p.client.I2PClientFactory
import net.i2p.client.I2PSession
import net.i2p.client.streaming.I2PSocketManager
import net.i2p.client.streaming.I2PSocketManagerFactory
import net.i2p.client.streaming.I2PSocketOptions
import net.i2p.client.streaming.I2PSocketManager.DisconnectListener
import net.i2p.crypto.DSAEngine
import net.i2p.crypto.SigType
import net.i2p.data.Destination
import net.i2p.data.PrivateKey
import net.i2p.data.Signature
@@ -100,6 +100,7 @@ public class Core {
final TrustService trustService
final TrustSubscriber trustSubscriber
private final PersisterService persisterService
private final PersisterFolderService persisterFolderService
private final HostCache hostCache
private final ConnectionManager connectionManager
private final CacheClient cacheClient
@@ -128,23 +129,23 @@ public class Core {
this.muOptions = props
i2pOptions = new Properties()
def i2pOptionsFile = new File(home,"i2p.properties")
// Read defaults
def defaultI2PFile = getClass()
.getClassLoader().getResource("defaults/i2p.properties");
defaultI2PFile.withInputStream { i2pOptions.load(it) }
def i2pOptionsFile = new File(home, "i2p.properties")
if (i2pOptionsFile.exists()) {
i2pOptionsFile.withInputStream { i2pOptions.load(it) }
if (!i2pOptions.containsKey("inbound.nickname"))
i2pOptions["inbound.nickname"] = "MuWire"
if (!i2pOptions.containsKey("outbound.nickname"))
i2pOptions["outbound.nickname"] = "MuWire"
} else {
i2pOptions["inbound.nickname"] = "MuWire"
i2pOptions["outbound.nickname"] = "MuWire"
i2pOptions["inbound.length"] = "3"
i2pOptions["inbound.quantity"] = "4"
i2pOptions["outbound.length"] = "3"
i2pOptions["outbound.quantity"] = "4"
i2pOptions["i2cp.tcp.host"] = "127.0.0.1"
i2pOptions["i2cp.tcp.port"] = "7654"
if (!i2pOptions.containsKey("outbound.nickname"))
i2pOptions["outbound.nickname"] = "MuWire"
}
if (!(i2pOptions.hasProperty("i2np.ntcp.port")
&& i2pOptions.hasProperty("i2np.udp.port")
)) {
Random r = new Random()
int port = r.nextInt(60000) + 4000
i2pOptions["i2np.ntcp.port"] = String.valueOf(port)
@@ -259,7 +260,14 @@ public class Core {
log.info "initializing persistence service"
persisterService = new PersisterService(new File(home, "files.json"), eventBus, 60000, fileManager)
eventBus.register(UILoadedEvent.class, persisterService)
eventBus.register(UIPersistFilesEvent.class, persisterService)
log.info "initializing folder persistence service"
persisterFolderService = new PersisterFolderService(this, new File(home, "files"), eventBus)
eventBus.register(PersisterDoneEvent.class, persisterFolderService)
eventBus.register(FileDownloadedEvent.class, persisterFolderService)
eventBus.register(FileLoadedEvent.class, persisterFolderService)
eventBus.register(FileHashedEvent.class, persisterFolderService)
eventBus.register(FileUnsharedEvent.class, persisterFolderService)
log.info("initializing host cache")
File hostStorage = new File(home, "hosts.json")
@@ -398,6 +406,8 @@ public class Core {
trustService.stop()
log.info("shutting down persister service")
persisterService.stop()
log.info("shutting down persisterFolder service")
persisterFolderService.stop()
log.info("shutting down download manager")
downloadManager.shutdown()
log.info("shutting down connection acceptor")

View File

@@ -0,0 +1,109 @@
package com.muwire.core.files
import com.muwire.core.DownloadedFile
import com.muwire.core.InfoHash
import com.muwire.core.Persona
import com.muwire.core.Service
import com.muwire.core.SharedFile
import com.muwire.core.util.DataUtil
import net.i2p.data.Base64
import net.i2p.data.Destination
import java.util.stream.Collectors
abstract class BasePersisterService extends Service{
protected static FileLoadedEvent fromJson(def json) {
if (json.file == null || json.length == null || json.infoHash == null || json.hashList == null)
throw new IllegalArgumentException()
if (!(json.hashList instanceof List))
throw new IllegalArgumentException()
def file = new File(DataUtil.readi18nString(Base64.decode(json.file)))
file = file.getCanonicalFile()
if (!file.exists() || file.isDirectory())
return null
long length = Long.valueOf(json.length)
if (length != file.length())
return null
List hashList = (List) json.hashList
ByteArrayOutputStream baos = new ByteArrayOutputStream()
hashList.each {
byte [] hash = Base64.decode it.toString()
if (hash == null)
throw new IllegalArgumentException()
baos.write hash
}
byte[] hashListBytes = baos.toByteArray()
InfoHash ih = InfoHash.fromHashList(hashListBytes)
byte [] root = Base64.decode(json.infoHash.toString())
if (root == null)
throw new IllegalArgumentException()
if (!Arrays.equals(root, ih.getRoot()))
return null
int pieceSize = 0
if (json.pieceSize != null)
pieceSize = json.pieceSize
if (json.sources != null) {
List sources = (List)json.sources
Set<Destination> sourceSet = sources.stream().map({ d -> new Destination(d.toString())}).collect Collectors.toSet()
DownloadedFile df = new DownloadedFile(file, ih, pieceSize, sourceSet)
df.setComment(json.comment)
return new FileLoadedEvent(loadedFile : df)
}
SharedFile sf = new SharedFile(file, ih, pieceSize)
sf.setComment(json.comment)
if (json.downloaders != null)
sf.getDownloaders().addAll(json.downloaders)
if (json.searchers != null) {
json.searchers.each {
Persona searcher = null
if (it.searcher != null)
searcher = new Persona(new ByteArrayInputStream(Base64.decode(it.searcher)))
long timestamp = it.timestamp
String query = it.query
sf.hit(searcher, timestamp, query)
}
}
return new FileLoadedEvent(loadedFile: sf)
}
protected static toJson(SharedFile sf) {
def json = [:]
json.file = sf.getB64EncodedFileName()
json.length = sf.getCachedLength()
InfoHash ih = sf.getInfoHash()
json.infoHash = sf.getB64EncodedHashRoot()
json.pieceSize = sf.getPieceSize()
json.hashList = sf.getB64EncodedHashList()
json.comment = sf.getComment()
json.hits = sf.getHits()
json.downloaders = sf.getDownloaders()
if (!sf.searches.isEmpty()) {
Set searchers = new HashSet<>()
sf.searches.each {
def search = [:]
if (it.searcher != null)
search.searcher = it.searcher.toBase64()
search.timestamp = it.timestamp
search.query = it.query
searchers.add(search)
}
json.searchers = searchers
}
if (sf instanceof DownloadedFile) {
json.sources = sf.sources.stream().map( {d -> d.toBase64()}).collect(Collectors.toList())
}
json
}
}

View File

@@ -6,4 +6,5 @@ import com.muwire.core.SharedFile
class FileLoadedEvent extends Event {
SharedFile loadedFile
String source
}

View File

@@ -0,0 +1,12 @@
package com.muwire.core.files
import com.muwire.core.Event
/**
* Should be triggered by the old PersisterService
* once it has finished reading the old file
*
* @see PersisterService
*/
class PersisterDoneEvent extends Event{
}

View File

@@ -0,0 +1,150 @@
package com.muwire.core.files
import com.muwire.core.*
import groovy.json.JsonOutput
import groovy.json.JsonSlurper
import groovy.util.logging.Log
import java.nio.file.Files
import java.nio.file.Path
import java.nio.file.Paths
import java.util.concurrent.ExecutorService
import java.util.concurrent.Executors
import java.util.concurrent.ThreadFactory
import java.util.logging.Level
/**
* A persister that stores information about the files shared using
* individual JSON files in directories.
*
* The absolute path's 32bit hash to the shared file is used
* to build the directory and filename.
*
* This persister only starts working once the old persister has finished loading
* @see PersisterFolderService#getJsonPath
*/
@Log
class PersisterFolderService extends BasePersisterService {
final static int CUT_LENGTH = 6
private final Core core;
final File location
final EventBus listener
final int interval
final Timer timer
final ExecutorService persisterExecutor = Executors.newSingleThreadExecutor({ r ->
new Thread(r, "file persister")
} as ThreadFactory)
PersisterFolderService(Core core, File location, EventBus listener) {
this.core = core;
this.location = location
this.listener = listener
this.interval = interval
timer = new Timer("file-folder persister timer", true)
}
void stop() {
timer.cancel()
persisterExecutor.shutdown()
}
void onPersisterDoneEvent(PersisterDoneEvent persisterDoneEvent) {
log.info("Old persister done")
load()
}
void onFileHashedEvent(FileHashedEvent hashedEvent) {
persistFile(hashedEvent.sharedFile)
}
void onFileDownloadedEvent(FileDownloadedEvent downloadedEvent) {
if (core.getMuOptions().getShareDownloadedFiles()) {
persistFile(downloadedEvent.downloadedFile)
}
}
/**
* Get rid of the json of unshared files
* @param unsharedEvent
*/
void onFileUnsharedEvent(FileUnsharedEvent unsharedEvent) {
def jsonPath = getJsonPath(unsharedEvent.unsharedFile)
def jsonFile = jsonPath.toFile()
if(jsonFile.isFile()){
jsonFile.delete()
}
}
void onFileLoadedEvent(FileLoadedEvent loadedEvent) {
if(loadedEvent.source == "PersisterService"){
log.info("Migrating persisted file from PersisterService: "
+ loadedEvent.loadedFile.file.absolutePath.toString())
persistFile(loadedEvent.loadedFile)
}
}
void load() {
log.fine("Loading...")
Thread.currentThread().setPriority(Thread.MIN_PRIORITY)
if (location.exists() && location.isDirectory()) {
try {
_load()
}
catch (IllegalArgumentException e) {
log.log(Level.WARNING, "couldn't load files", e)
}
} else {
location.mkdirs()
listener.publish(new AllFilesLoadedEvent())
}
loaded = true
}
/**
* Loads every JSON into memory
*/
private void _load() {
int loaded = 0
def slurper = new JsonSlurper()
Files.walk(location.toPath())
.filter({ it.fileName.endsWith(".json") })
.forEach({
def parsed = slurper.parse it.toFile()
def event = fromJson parsed
if (event == null) return
log.fine("loaded file $event.loadedFile.file")
listener.publish event
loaded++
if (loaded % 10 == 0)
Thread.sleep(20)
})
listener.publish(new AllFilesLoadedEvent())
}
private void persistFile(SharedFile sf) {
persisterExecutor.submit({
def jsonPath = getJsonPath(sf)
def startTime = System.currentTimeMillis()
jsonPath.parent.toFile().mkdirs()
jsonPath.toFile().withPrintWriter { writer ->
def json = toJson sf
json = JsonOutput.toJson(json)
writer.println json
}
log.fine("Time(ms) to write json: " + (System.currentTimeMillis() - startTime))
} as Runnable)
}
private Path getJsonPath(SharedFile sf){
def pathHash = sf.getB64PathHash()
return Paths.get(
location.getAbsolutePath(),
pathHash.substring(0, CUT_LENGTH),
pathHash.substring(CUT_LENGTH) + ".json"
)
}
}

View File

@@ -1,40 +1,24 @@
package com.muwire.core.files
import java.nio.file.CopyOption
import java.nio.file.Files
import java.nio.file.StandardCopyOption
import java.util.concurrent.ExecutorService
import java.util.concurrent.Executors
import java.util.concurrent.ThreadFactory
import java.util.logging.Level
import java.util.stream.Collectors
import com.muwire.core.DownloadedFile
import com.muwire.core.EventBus
import com.muwire.core.InfoHash
import com.muwire.core.Persona
import com.muwire.core.Service
import com.muwire.core.SharedFile
import com.muwire.core.UILoadedEvent
import com.muwire.core.util.DataUtil
import groovy.json.JsonOutput
import groovy.json.JsonSlurper
import groovy.util.logging.Log
import net.i2p.data.Base64
import net.i2p.data.Destination
@Log
class PersisterService extends Service {
class PersisterService extends BasePersisterService {
final File location
final EventBus listener
final int interval
final Timer timer
final FileManager fileManager
final ExecutorService persisterExecutor = Executors.newSingleThreadExecutor({ r ->
new Thread(r, "file persister")
} as ThreadFactory)
PersisterService(File location, EventBus listener, int interval, FileManager fileManager) {
this.location = location
@@ -51,10 +35,6 @@ class PersisterService extends Service {
void onUILoadedEvent(UILoadedEvent e) {
timer.schedule({load()} as TimerTask, 1)
}
void onUIPersistFilesEvent(UIPersistFilesEvent e) {
persistFiles()
}
void load() {
Thread.currentThread().setPriority(Thread.MIN_PRIORITY)
@@ -69,6 +49,7 @@ class PersisterService extends Service {
def event = fromJson parsed
if (event != null) {
log.fine("loaded file $event.loadedFile.file")
event.source = "PersisterService"
listener.publish event
loaded++
if (loaded % 10 == 0)
@@ -76,126 +57,18 @@ class PersisterService extends Service {
}
}
}
listener.publish(new AllFilesLoadedEvent())
} catch (IllegalArgumentException|NumberFormatException e) {
// Backup the old hashes
location.renameTo(
new File(location.absolutePath + ".bak")
)
listener.publish(new PersisterDoneEvent())
} catch (IllegalArgumentException e) {
log.log(Level.WARNING, "couldn't load files",e)
}
} else {
listener.publish(new AllFilesLoadedEvent())
listener.publish(new PersisterDoneEvent())
}
timer.schedule({persistFiles()} as TimerTask, 1000, interval)
loaded = true
}
private static FileLoadedEvent fromJson(def json) {
if (json.file == null || json.length == null || json.infoHash == null || json.hashList == null)
throw new IllegalArgumentException()
if (!(json.hashList instanceof List))
throw new IllegalArgumentException()
def file = new File(DataUtil.readi18nString(Base64.decode(json.file)))
file = file.getCanonicalFile()
if (!file.exists() || file.isDirectory())
return null
long length = Long.valueOf(json.length)
if (length != file.length())
return null
List hashList = (List) json.hashList
ByteArrayOutputStream baos = new ByteArrayOutputStream()
hashList.each {
byte [] hash = Base64.decode it.toString()
if (hash == null)
throw new IllegalArgumentException()
baos.write hash
}
byte[] hashListBytes = baos.toByteArray()
InfoHash ih = InfoHash.fromHashList(hashListBytes)
byte [] root = Base64.decode(json.infoHash.toString())
if (root == null)
throw new IllegalArgumentException()
if (!Arrays.equals(root, ih.getRoot()))
return null
int pieceSize = 0
if (json.pieceSize != null)
pieceSize = json.pieceSize
if (json.sources != null) {
List sources = (List)json.sources
Set<Destination> sourceSet = sources.stream().map({d -> new Destination(d.toString())}).collect Collectors.toSet()
DownloadedFile df = new DownloadedFile(file, ih, pieceSize, sourceSet)
df.setComment(json.comment)
return new FileLoadedEvent(loadedFile : df)
}
SharedFile sf = new SharedFile(file, ih, pieceSize)
sf.setComment(json.comment)
if (json.downloaders != null)
sf.getDownloaders().addAll(json.downloaders)
if (json.searchers != null) {
json.searchers.each {
Persona searcher = null
if (it.searcher != null)
searcher = new Persona(new ByteArrayInputStream(Base64.decode(it.searcher)))
long timestamp = it.timestamp
String query = it.query
sf.hit(searcher, timestamp, query)
}
}
return new FileLoadedEvent(loadedFile: sf)
}
private void persistFiles() {
persisterExecutor.submit( {
def sharedFiles = fileManager.getSharedFiles()
File tmp = File.createTempFile("muwire-files", "tmp")
tmp.deleteOnExit()
tmp.withPrintWriter { writer ->
sharedFiles.each { k, v ->
def json = toJson(k,v)
json = JsonOutput.toJson(json)
writer.println json
}
}
Files.copy(tmp.toPath(), location.toPath(), StandardCopyOption.REPLACE_EXISTING)
tmp.delete()
} as Runnable)
}
private def toJson(File f, SharedFile sf) {
def json = [:]
json.file = sf.getB64EncodedFileName()
json.length = sf.getCachedLength()
InfoHash ih = sf.getInfoHash()
json.infoHash = sf.getB64EncodedHashRoot()
json.pieceSize = sf.getPieceSize()
json.hashList = sf.getB64EncodedHashList()
json.comment = sf.getComment()
json.hits = sf.getHits()
json.downloaders = sf.getDownloaders()
if (!sf.searches.isEmpty()) {
Set searchers = new HashSet<>()
sf.searches.each {
def search = [:]
if (it.searcher != null)
search.searcher = it.searcher.toBase64()
search.timestamp = it.timestamp
search.query = it.query
searchers.add(search)
}
json.searchers = searchers
}
if (sf instanceof DownloadedFile) {
json.sources = sf.sources.stream().map( {d -> d.toBase64()}).collect(Collectors.toList())
}
json
}
}

View File

@@ -1,6 +0,0 @@
package com.muwire.core.files
import com.muwire.core.Event
class UIPersistFilesEvent extends Event {
}

View File

@@ -2,6 +2,8 @@ package com.muwire.core;
import java.io.File;
import java.io.IOException;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.util.ArrayList;
import java.util.Collections;
import java.util.HashSet;
@@ -21,7 +23,8 @@ public class SharedFile {
private final String cachedPath;
private final long cachedLength;
private String b64PathHash;
private final String b64EncodedFileName;
private final String b64EncodedHashRoot;
private final List<String> b64EncodedHashList;
@@ -38,7 +41,7 @@ public class SharedFile {
this.cachedLength = file.length();
this.b64EncodedFileName = Base64.encode(DataUtil.encodei18nString(file.toString()));
this.b64EncodedHashRoot = Base64.encode(infoHash.getRoot());
List<String> b64List = new ArrayList<String>();
byte[] tmp = new byte[32];
for (int i = 0; i < infoHash.getHashList().length / 32; i++) {
@@ -52,6 +55,19 @@ public class SharedFile {
return file;
}
public byte[] getPathHash() throws NoSuchAlgorithmException {
var digester = MessageDigest.getInstance("SHA-256");
digester.update(file.getAbsolutePath().getBytes());
return digester.digest();
}
public String getB64PathHash() throws NoSuchAlgorithmException {
if(b64PathHash == null){
b64PathHash = Base64.encode(getPathHash());
}
return b64PathHash;
}
public InfoHash getInfoHash() {
return infoHash;
}

View File

@@ -0,0 +1,8 @@
inbound.nickname=MuWire
outbound.nickname=MuWire
inbound.length=3
inbound.quantity=4
outbound.length=3
outbound.quantity=4
i2cp.tcp.host=127.0.0.1
i2cp.tcp.port=7654

View File

@@ -0,0 +1,26 @@
#!/usr/bin/with-contenv sh
#
# Add the app user to the password and group databases. This is needed just to
# make sure that mapping between the user/group ID and its name is possible.
#
set -e # Exit immediately if a command exits with a non-zero status.
set -u # Treat unset variables as an error.
cp /defaults/passwd /etc/passwd
cp /defaults/group /etc/group
cp /defaults/shadow /etc/shadow
chown root:shadow /etc/shadow
chmod 640 /etc/shadow
echo "$APP_USER:x:$USER_ID:$GROUP_ID::${APP_HOME:-/dev/null}:/sbin/nologin" >> /etc/passwd
echo "$APP_USER:x:$GROUP_ID:" >> /etc/group
# Make sure APP_HOME is editable by the user
if [[ -n "$APP_HOME" ]] ; then
chown -R "$APP_USER" "$APP_HOME"
chmod -R u+rw "$APP_HOME"
fi
# vim:ft=sh:ts=4:sw=4:et:sts=4

View File

@@ -0,0 +1,34 @@
#This file is UTF-8
#Tue Jan 14 12:08:47 GMT 2020
meshExpiration=60
autoDownloadUpdate=true
hostHopelessInterval=1440
uploadSlotsPerUser=-1
downloadLocation=/output
allowTrustLists=true
embeddedRouter=false
incompleteLocation=/incompletes
outBw=128
searchExtraHop=false
shareHiddenFiles=false
advertiseChat=true
totalUploadSlots=-1
hostClearInterval=15
searchComments=true
downloadSequentialRatio=0.8
maxChatConnectios=-1
trustListInterval=1
crawlerResponse=REGISTERED
browseFiles=true
lastUpdateCheck=1579003533112
hostRejectInterval=1
inBw=256
leaf=false
updateCheckInterval=24
plugin=false
downloadRetryInterval=60
speedSmoothSeconds=60
allowUntrusted=true
shareDownloadedFiles=true
startChatServer=false
updateType=jar

View File

@@ -0,0 +1 @@
i2cp.tcp.host=172.17.0.1

View File

@@ -0,0 +1,7 @@
#!/bin/sh
# Explicitly define HOME otherwise it might not have been set
export HOME=/muwire
echo "Starting MuWire"
exec /muwire/bin/MuWire

View File

@@ -3,15 +3,11 @@ package com.muwire.gui
import griffon.core.GriffonApplication
import griffon.core.artifact.GriffonController
import griffon.core.controller.ControllerAction
import griffon.core.mvc.MVCGroup
import griffon.core.mvc.MVCGroupConfiguration
import griffon.inject.MVCMember
import griffon.metadata.ArtifactProviderFor
import groovy.json.StringEscapeUtils
import net.i2p.crypto.DSAEngine
import net.i2p.data.Base64
import net.i2p.data.Signature
import net.i2p.data.SigningPrivateKey
import java.awt.Desktop
import java.awt.Toolkit
@@ -30,15 +26,11 @@ import com.muwire.core.Persona
import com.muwire.core.SharedFile
import com.muwire.core.SplitPattern
import com.muwire.core.download.Downloader
import com.muwire.core.download.DownloadStartedEvent
import com.muwire.core.download.UIDownloadCancelledEvent
import com.muwire.core.download.UIDownloadEvent
import com.muwire.core.download.UIDownloadPausedEvent
import com.muwire.core.download.UIDownloadResumedEvent
import com.muwire.core.filecert.UICreateCertificateEvent
import com.muwire.core.files.DirectoryUnsharedEvent
import com.muwire.core.files.FileUnsharedEvent
import com.muwire.core.files.UIPersistFilesEvent
import com.muwire.core.search.QueryEvent
import com.muwire.core.search.SearchEvent
import com.muwire.core.trust.RemoteTrustList
@@ -371,7 +363,6 @@ class MainFrameController {
sf.each {
core.eventBus.publish(new FileUnsharedEvent(unsharedFile : it))
}
core.eventBus.publish(new UIPersistFilesEvent())
}
@ControllerAction
@@ -534,4 +525,4 @@ class MainFrameController {
core = e.getNewValue()
})
}
}
}

View File

@@ -44,6 +44,8 @@ class Ready extends AbstractLifecycleHandler {
propsFile.withReader("UTF-8", {
props.load(it)
})
if (!props.containsKey("nickname"))
props.setProperty("nickname", selectNickname())
props = new MuWireSettings(props)
if (props.incompleteLocation == null)
props.incompleteLocation = new File(home, "incompletes")
@@ -53,29 +55,7 @@ class Ready extends AbstractLifecycleHandler {
props.incompleteLocation = new File(home, "incompletes")
props.embeddedRouter = Boolean.parseBoolean(System.getProperties().getProperty("embeddedRouter"))
props.updateType = System.getProperty("updateType","jar")
def nickname
while (true) {
nickname = JOptionPane.showInputDialog(null,
"Your nickname is displayed when you send search results so other MuWire users can choose to trust you",
"Please choose a nickname", JOptionPane.PLAIN_MESSAGE)
if (nickname == null) {
JOptionPane.showMessageDialog(null, "MuWire cannot start without a nickname and will now exit", JOptionPane.PLAIN_MESSAGE)
System.exit(0)
}
if (nickname.trim().length() == 0) {
JOptionPane.showMessageDialog(null, "Nickname cannot be empty", "Select another nickname",
JOptionPane.WARNING_MESSAGE)
continue
}
if (nickname.contains("@")) {
JOptionPane.showMessageDialog(null, "Nickname cannot contain @, choose another",
"Select another nickname", JOptionPane.WARNING_MESSAGE)
continue
}
nickname = nickname.trim()
break
}
props.setNickname(nickname)
props.setNickname(selectNickname())
def portableDownloads = System.getProperty("portable.downloads")
@@ -120,5 +100,31 @@ class Ready extends AbstractLifecycleHandler {
core.eventBus.publish(new UILoadedEvent())
}
private String selectNickname() {
String nickname
while (true) {
nickname = JOptionPane.showInputDialog(null,
"Your nickname is displayed when you send search results so other MuWire users can choose to trust you",
"Please choose a nickname", JOptionPane.PLAIN_MESSAGE)
if (nickname == null) {
JOptionPane.showMessageDialog(null, "MuWire cannot start without a nickname and will now exit", JOptionPane.PLAIN_MESSAGE)
System.exit(0)
}
if (nickname.trim().length() == 0) {
JOptionPane.showMessageDialog(null, "Nickname cannot be empty", "Select another nickname",
JOptionPane.WARNING_MESSAGE)
continue
}
if (nickname.contains("@")) {
JOptionPane.showMessageDialog(null, "Nickname cannot contain @, choose another",
"Select another nickname", JOptionPane.WARNING_MESSAGE)
continue
}
nickname = nickname.trim()
break
}
nickname
}
}

File diff suppressed because one or more lines are too long

After

Width:  |  Height:  |  Size: 37 KiB

BIN
images/i2cp_config.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 13 KiB