Open
Description
I would love to see if someone could add jq and json2csv as I have a repeated workflow that include these tools. Thank you.
I would love to see if someone could add jq and json2csv as I have a repeated workflow that include these tools. Thank you.
Activity
fprimex commentedon Jun 3, 2022
I have
jq
working in the simulator and am trying to finish up getting it added. I'm not familiar withjson2csv
. Do you have a link to it?hhhuff commentedon Jun 4, 2022
fprimex commentedon Jun 4, 2022
I had a quick look and the difficulty here is that the program is written in Go. This language toolset is not yet working with ios_system. I compiled it and it produces neither output nor files.
Go can natively produce both wasm and ios/arm64 formats, however the wasm SDK is not the one needed for to produce compatibly ios_system output.
The ios/arm64 output would work, but Go does not support outputting dynamic, shared libraries on ios/arm64. I tried combinations of
-buildmode=shared
,buildmode=c-shared
, and-linkshared
but none were accepted.Honestly, your best bet for getting json2csv is to port it to python or jq. jq has an
@csv
output target. I would like to see Go work with ios_system at some point, but I feel like that is probably a long ways off.fprimex commentedon Jun 4, 2022
I would be interested in decompiling the resulting Go wasm, replacing the needed SDK calls, and then reassembling it. That would enable Go programs in
ios_system
. I'll try that later.fprimex commentedon Jun 4, 2022
Used
wabt
to convert the wasm to c. It was over 32MB and 1M lines long 😅. It must have the entire Go stdlib in it. I tried compiling it, but killed it when it had consumed 18GB of RAM on my system.holzschu commentedon Jun 5, 2022
I've been thinking. WASM should be portable (it is designed for that). The only potentially non-portable parts are when it tries to access the file system, which is usually encapsulated into JavaScript commands. What kind of errors do you get with
json2csv.wasm
?fprimex commentedon Jun 5, 2022
IIRC it just doesn't work. No output, no errors, no files.
holzschu commentedon Jun 5, 2022
That doesn't make a lot of clues, I'll grant you that. But normally, a webAssembly command that does not execute produces an error, so "no errors" could be a positive sign. Since webAssembly does not have access to the file system, I would try with stdin and stdout (
json2csv < input.json > output.csv
).fprimex commentedon Jun 5, 2022
This similarly produces no output.
fprimex commentedon Jun 6, 2022
Go wasm does not provide wasi and therefore cannot be used in the context ios_system / a-shell wants. The
tinygo
provider, however, does and compiling with it works fine. Unfortunately,tinygo
does not support everything that Go does, and the resulting wasm "panics" with areflect.NumMethod
not implemented error. This is apparently common fortinygo
programs usingencoding/json
.I'm looking at one other option and then I think I'm out of ideas. The good news is that using
jq
to do this would be relatively simple.holzschu commentedon Jun 6, 2022
I haven't looked at the Go documentation, but if their WebAssembly version comes with a JavaScript container, maybe we can run this JS container rather than ours?
jsc
lets you run JavaScript commands, and the code a-Shell uses to run WebAssembly is inwasm.js
. If it works, we will have to design a way to switch between JS containers depending on the WebAssembly flavour.fprimex commentedon Jun 8, 2022
Here is the working
wasm
file forjson2csv
to place inDocuments/bin
. GH made me zip it so I could upload it 🙄 , so it'll need to be unzipped as well.json2csv.wasm.zip
holzschu commentedon Jun 9, 2022
Thanks for compiling this command.
I've made it available on https://github.com/holzschu/a-Shell-commands/ but when I test, I get blank output every time. I might be using it wrong, though.
fprimex commentedon Jun 9, 2022
I think that version is old and broken. I have attached a tar.gz with the wasm and some tests that can be run that are from the project's acceptance tests and docs. See the tests.txt file for what to run and the expected output (there will be some newline representation differences because of how I copied the tests over most likely).
json2csv_wasm.tar.gz
6 remaining items