Today we are going to create a FIFO (First In First Out) type file stream between the browser and a local application.
The idea is to create a local file in the browser, when the file is created the local application writes data to the newly created file, in the browser we will read the data written to the file up to that point, then truncate the file to size 0, repeat, when the writing and reading of the file are complete we will remove the file from the local file system.
In this way we can configure our local application to watch for the creation of specific named files, each performing a discrete process, and stream data from the local application to the file, without using networking.
We'll be using Deno in this example for the local application. Use whatever local application you want. Node.js, QuickJS, C, C++, Rust, Bash, whatever.
Our JavaScript code looks something like this watcher.js
// deno run -A watcher.js
const file = 'output.txt';
while (true) {
console.log(`Watching ${file}`);
const watcher = Deno.watchFs('');
try {
for await (const {
kind,
paths: [path]
}
of watcher) {
if (path.split('/').pop() === file && kind === 'access') {
console.log(kind, path);
break;
}
}
} catch {}
try {
watcher.close();
} catch {}
let data = 'abcdefghijklmnopqrstuvwxyz';
for (const char of data) {
console.log(char);
const handle = await Deno.open(file, {
read: true,
write: true
});
await new Blob([char.toUpperCase().repeat(441 * 4)], {
type: 'application/octet-stream'
})
.stream().pipeTo(handle.writable);
// Wait for the file to be read and truncated in the browser
while (true) {
const {
size
} = await Deno.stat(file);
if (size === 0) {
break;
}
}
await new Promise((resolve) => setTimeout(resolve, 60));
}
await Deno.remove(file);
console.log('Done file streaming');
}
We are keeping the script running in the while
loop so we can create, write and read to a file multiple times without
stopping and starting the local script.
Synchronizing file creation, writing and reading the created file between the local application and the browser is not
trivial. For now we use a 60 millisecond delay after the file has been written to by the local application, then after we
get the file information where the file size is 0 before moving on to the next iteration of the loop which writes data to
file. Technically the file will be removed then re-written by File System Access API when truncate(0)
is called.
In the browser (Chromium 118) with --enable-features=FileSystemObserver
flag set we do something like this in fso.js
var {
readable,
writable
} = new TransformStream();
var handle = await showSaveFilePicker({
multiple: 'false',
types: [{
description: "Stream",
accept: {
"application/octet-stream": [".txt"],
},
}, ],
excludeAcceptAllOption: true,
startIn: 'documents',
suggestedName: 'output.txt'
});
var status = await handle.requestPermission({
mode: "readwrite"
});
var fileStream = async (changedHandle, type = '') => {
try {
var {
size
} = await changedHandle.getFile();
if (size) {
var stream = await changedHandle.createWritable();
var file = await changedHandle.getFile();
await file.stream().pipeTo(writable, {
preventClose: true
});
await stream.truncate(0);
await stream.close();
}
} catch (e) {
console.log(e);
try {
fso.unobserve(changedHandle);
await writable.close();
} catch (err) {
console.log(err);
}
finally {
console.log(handle);
await handle.remove();
}
}
}
readable.pipeThrough(new TextDecoderStream()).pipeTo(
new WritableStream({
async start() {
try {
return fileStream(handle);
} catch (e) {
console.log(e);
throw e;
}
},
write(value) {
console.log(value.length);
},
close() {
console.log('Stream closed');
}
})
).catch(console.error);
var fso = new FileSystemObserver(async ([{
changedHandle,
root,
type
}], record) => {
try {
await fileStream(changedHandle, type);
} catch (e) {
console.log(e);
}
});
fso.observe(handle);
Notice the first call to fileStream(handle)
in the start()
method of the WritableStream
we are piping our data to,
so we don't miss the data written to the file on the first iteration of the loop where we write data.
If all went as expected we should begin with no file named output.txt
on the file system, create the file in the browser,
upon creation the file watching application (notify()
, inotify-tools
, etc.) begins writing data, waiting for the data to
be read then the file to be truncated in the browser, then cycles to the next iteration of writing data to the file, and
when the process is complete the file output.txt
will be removed from the filesystem. We try to make sure of the removal
by redundantly using Deno.remove()
in the local application and handle.remove()
in the browser, where handle
is an
instance of a FileSystemFileHandle
.