The plugin does not work on OSX #11
Comments
|
Hi Mauricio, I'd be happy to help. Can you give me instructions to compile the plugin on OSX? |
|
Excellent! Let's do some checks first.
Iet me know what happens. You only need to specify |
|
Do I need to compile in the trunk? |
|
I think it would be best if you made a branch for it. Maybe run Before doing anything. |
|
I just checked out the entire repository using -svn co-. |
|
You can use SVN with github? I didn't know that... I don't know enough SVN to help with branches. I think you will need git for |
|
Ok. I realized that git comes with Xcode. I cloned the rep and started a new branch using git checkout -b osx but I have issues with premake5. See below: Federicos-MacBook-Pro:stata-gtools federico$ make spooky PREMAKE=/Users/federico/Desktop/GiT/premake5/premake5.lua |
|
Is |
|
It is a script. But I also followed the instructions at https://github.com/premake/premake-core/wiki/Building-Premake to create the Federicos-MacBook-Pro:stata-gtools federico$ make spooky PREMAKE=/Users/federico/Desktop/GiT/premake5/bin/release/premake5 |
|
That's odd. Can you run cd lib/spookyhash/build
ls -lah
/Users/federico/Desktop/GiT/premake5/bin/release/premake5 gmake make |
|
Federicos-MacBook-Pro:stata-gtools federico$ cd lib/spookyhash/build |
|
Oh! I know what's up. Totally my bad; I should have read my own README. You need git submodule update --init --recursivebefore doing anything else. Try that and try the previous set of commands again. |
|
Federicos-MacBook-Pro:GiT federico$ git clone https://github.com/mcaceresb/stata-gtools.git stata-gtools Federicos-MacBook-Pro:GiT federico$ cd stata-gtools/ Federicos-MacBook-Pro:stata-gtools federico$ git checkout -b osx Federicos-MacBook-Pro:stata-gtools federico$ make spooky PREMAKE=/Users/federico/Desktop/GiT/premake5/bin/release/premake5 Error: ...sktop/GiT/stata-gtools/lib/spookyhash/build/premake5.lua:37: attempt to compare number with boolean make: *** [spooky] Error 1 |
|
Please delete lines 37 through 40 of |
|
line 37 in the ../stata-gtools/lib/spookyhash/build/premake5.lua script looks if os.execute("git --version") > 0 then |
|
Ok. A step ahead...but still an error Federicos-MacBook-Pro:stata-gtools federico$ make spooky PREMAKE=/Users/federico/Desktop/GiT/premake5/bin/release/premake5 ** Warning: the flags value OptimizeSpeed has been deprecated and will be removed. Building configurations... |
|
Ok, but that's great progress, actually. Can you tell me the output of |
|
You're looking for the library it made. I thought it would have the same name as in Linux but it seems not. |
|
Federicos-MacBook-Pro:stata-gtools federico$ ls -lahR /Users/federico/Desktop/GiT/stata-gtools/lib/spookyhash/build/ /Users/federico/Desktop/GiT/stata-gtools/lib/spookyhash/build//bin: /Users/federico/Desktop/GiT/stata-gtools/lib/spookyhash/build//bin/Release: /Users/federico/Desktop/GiT/stata-gtools/lib/spookyhash/build//obj: /Users/federico/Desktop/GiT/stata-gtools/lib/spookyhash/build//obj/Release: /Users/federico/Desktop/GiT/stata-gtools/lib/spookyhash/build//obj/Release/spookyhash-shared: /Users/federico/Desktop/GiT/stata-gtools/lib/spookyhash/build//obj/Release/spookyhash-static: /Users/federico/Desktop/GiT/stata-gtools/lib/spookyhash/build//obj/Release/spookyhash-test: |
|
That looks excellent! Thanks. Ok, let's try this: make SPOOKYLIB=-l:libspookyhash.aif that fails, try this gcc -Wall -bundle -DSYSTEM=APPLEMAC -c -o build/stplugin.o src/plugin/spi/stplugin.c
gcc -Wall -bundle -DSYSTEM=APPLEMAC -c -o build/gtools.o src/plugin/gtools.c
gcc -Wall -bundle -DSYSTEM=APPLEMAC -o build/gtools_macosx.plugin build/gtools.o build/stplugin.o -L./lib/spookyhash/build/bin/Release -l:libspookyhash.a
gcc -Wall -bundle -DSYSTEM=APPLEMAC -o build/env_set_macosx.plugin src/plugin/spi/stplugin.c src/plugin/env_set.cIf both fail, please post the output. |
|
The The same is true for the |
|
I think it might be the double slash. Hopefully this works: If not, can you go into |
|
I finally found the right syntax to link the library using
After adding the missing line
in the gtools.pkg file I tried to reinstall the
and it seems that now the only missing file is the gtools_macosx_legacy.plugin. What do I have to compile to get this file? |
|
Ah, excellent! I've created an osx-plugin repo. Please fork the repo and compile the plugin on that branch.
cd /Users/federico/Desktop/GiT/
git clone https://github.com/fbelotti/stata-gtools stata-gtools-fork
cd stata-gtools-fork
git submodule update --init --recursive
git checkout osx-plugin
make spooky PREMAKE=/Users/federico/Desktop/GiT/premake5/bin/release/premake5
make
cd /Users/federico/Desktop/GiT/stata-gtools-fork/build
which gtoolsMake sure the version is 0.7.3. Now run log using gtools_tests_macosx.log, text
do gtools_tests.do
log close _all
cd /Users/federico/Desktop/GiT/stata-gtools-fork/
mv build/gtools_tests_macosx.log src/test/
git add src/test/gtools_tests_macosx.log
git add lib/plugin/gtools_macosx.plugin
git add lib/plugin/env_set_macosx.plugin
git add build/gtools_macosx.plugin -f
git add build/env_set_macosx.plugin -f
git commit -m "Compiled OSX plugin; tests passing."
git push -u origin osx-plugin
Let me know if anything goes array. Mauricio |
|
Hi Mauricio Unfortunately there still are some issues. I did it, and the process goes well up to
When I run Any hint? Federico |
|
Try copying If that works I'll modify the code to use the dynamic library for OSX as well. |
|
No good news. The tests fail with the same error. |
|
The |
|
There should be a copy in |
|
Ok, I copied from lib/plugin. The output of |
|
Ok, that's what I expected. Great. Now can you try running step 2? So, via stata-mp, dii "`c(machine_type)'"
program gtools_plugin, plugin using(`"gtools_macosx.plugin"')
plugin call gtools_plugin, checkIf that all works, please replace line 1409 in sysuse auto
gcollapse price, by(make) v bfrom |
|
The first part runs without errors. After replacing the code in The result printed to the shell is Attached you can see the full log. I also tried running the same commands in the Stata gui and it still crashes. |
|
Ok, it seems that's a memory error, so it may be that I'm not being careful about allocating all the internal arrays and I just haven't noticed because GCC in other systems is trying to fix it for me. If this is the case it's very fixable. Let's check:
gcc -Wall -bundle -DSYSTEM=APPLEMAC lib/spi-2.0/stplugin.c test/spooky_test.c lib/spookyhash/build/bin/Release/libspookyhash.a -o test/spooky_test.plugin
program spooky_test, plugin using(spooky_test.plugin)
sysuse auto
plugin call spooky_test make price, check
plugin call spooky_test make price, read
plugin call spooky_test make price, spookyand run it via |
|
Ok, that's good news. There isn't a bug with spookyhash or calling it from Stata. The problem is the memory allocation. This means somewhere I define some string or array to be shorter than Stata expects it to be... I will go through my code and double check everything. By the way, do the following run? (From the sysuse auto
gcollapse price, by(foreign) v b
gcollapse foreign, by(price) v bThe internal code handles integers, floats and strings differently so this will give me a clue as to where my mistake is. Thanks a lot for helping narrow down the issue. |
|
You are welcome! This helps me a lot in understanding how Stata plugin works. Attached the log file. |
|
I'll be pushing the new version tomorrow. I've made it so every array is allocated, so hopefully there won't be any memory leaks. Before I do that, I want to try and fix the mishap where Stata in batch mode thinks it is in Unix. Can you tell me what happens when you run this from the command line (i.e. using di "`c(machine_type)'" |
|
Great. This is what the machine_type macro looks like running stata-mp from terminal |
|
Ok. I have taken this into account. Please run git fetch upstream
git checkout osx-plugin
git merge upstream/osx-pluginAnd then re-compile the plugin and try to run the tests again ( |
|
@fbelotti Just wanted to follow-up and see if you'd had time to re-try compiling the plugin and running the tests. |
|
Hi Mauricio, sorry I didn't see your email. |
|
I always forget you have to add remotes. Here: Then re-run the commands. If there are merge conflicts on .ado, .c, .do, or .sthlp files, just accept the ones from upstream. If there are merge conflicts in other files let me know. |
|
It seems that the git |
|
Perhaps it is due to the changes. Let's see: git commit -a -m "Working draft of OSX plugin (0.7.3)"
git fetch upstream
git merge upstream/osx-plugin? |
|
Hi Mauricio. I think I need to refresh my working copy. There is something wrong. Do I need to fork again from the rep? |
|
Hi; apologies I have been AWAL. I decided to refactor the code base, which took some a while since I could only work on this on my spare time. Anyway, could you start fresh? Several new commands and options have been added since we first started this thread. git clone https://github.com/mcaceresb/stata-gtools
cd stata-gtools
git checkout osx-plugin
git submodule update --init --recursive
sed -i.bak -e '37,40d' ./lib/spookyhash/build/premake5.lua
make spooky PREMAKE=/Users/federico/Desktop/GiT/premake5/bin/release/premake5
make clean
makeIf that works, go into if the first |
|
Travis does not have Stata, which is why I don't use it in this way. You will note that my .travis.yml file only runs make. The reason is that I only commit after running the set of tests I've written (and usually after pushing I install from the repo and re-run the tests just to double-check). While I might forgo the second test, I don't think it's sensible to commit w/o testing the plugin at all because even a very minor mistake could cause a crash. (As an aside, when did Travis add Windows support? I use appveyor to test windows bc I thought Travis only supported OSX and Linux.) |
|
Sorry, it is indeed appveyor for Windows. I agree that committing without testing is useful. One option then is to use git hooks to run the master do file with the tests? One problem I envision is that it would be good that the three OSes have the same version, which means waiting for OSX users to run the update. This can be solved if you use Travis for the OSX build, with the caveat that errors that occur on OSX only will not be detected. |
|
If I manage to get OSX working and gtools to The OSX version already compiles and loads w/o error, btw, but the latest was that it crashed Stata; pretty sure I fixed that but it's wait and see for now. So I'm really uneasy about committing at all w/o running tests. I'd be happy to use travis if I knew I could get the tests running elsewhere, but I don't see how git hooks solves this. I still need OSX w Stata for that. |
gtools-0.9.0 (2017-11-01) Features - The plugin now works on OSX - Online documentation is now available: [gtools.readthedocs.io](http://gtools.readthedocs.io/en/latest/index.html) - `gcontract` is a fast alternative to `contrast` - `gtoplevelsof` is a new command that allows the user to glean the most common levels of a set of variables. Similar to `gcontract` with a `gsort` of frequency by descending order thereafter, but `gtoplevelsof` does not modify the source data and saves a matrix with the results after its run. - `gdistinct` now saves its results to a matrix when there are multiple variables. - Improved and normalized documentation Bug fixes - OSX version; fixes #11 - `gisid` now sient w/o benchmark or verbose; fixes #20 - Added quotes to `cd cwd` in `gtools`; fixes #22 - `gcontract` available; fixes #23
|
@fbelotti : Thanks for all your help debugging this. I was able to compile on a MacbookAir I was lent with all tests passing, so I closed the issue. 0.9.0 in master should include a full-featured OSX version. |
Since I do not have a Mac and I have been unable to install OSX on a virtual machine, I have not been able to compile an OSX version of the plugin. This is currently on hold. Macs are expensive.
If you happen to have a Mac and are willing yo contribute to gtools, please comment here and I will give you instructions on how to compile.
The text was updated successfully, but these errors were encountered: