Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Windows support for SerialPort #3809

Merged
merged 42 commits into from
Aug 7, 2018
Merged

Windows support for SerialPort #3809

merged 42 commits into from
Aug 7, 2018

Conversation

mossheim
Copy link
Contributor

@mossheim mossheim commented Jun 24, 2018

This PR adds Windows support for SerialPort and reworks the entire SerialPort class. Fixes #1008.

Based most of this off @timblechmann 's prior work.

Since we are now introducing this feature on a new platform, I thought this would be a good time to update the API/behavior a bit. I'm pretty sure I've done this in a non-breaking way for sane usage.

The main changes are:

  • deprecated cleanupAll, no longer needed (see below)
  • deprecated timeout arguments to put and putAll - this just seems like bad design to me. put will fail if there is an error writing; I don't think it's a good idea to go into an infinite loop trying to write at that point.
  • internally, use isOpen variable instead of the dataptr directly to determine open status (makes shutdown simpler)
  • remove the default doneAction - in general I think we're trying to cut down on overbearing posts.
  • reworked shutdown procedure

When close is triggered, a complex sequence of events happens:

  1. The C++ port is closed and the SerialPort object is marked as closed. The C++ port's read thread waits on the global sclang execution lock to execute prDoneAction
  2. The SC thread running close eventually yields, allowing the SerialPort's prDoneAction to run on a separate thread
  3. That prDoneAction causes a deferred call to prCleanup. At this point, not deferring would lead to a crash, because the C++ object's destructor needs its read thread to join, which would mean we are trying to join a thread from that thread itself - deadlock. This SC thread yields.
  4. prCleanup sets the dataptr to nil and destroys the C++ object, and the read thread can now safely join.

I don't know if there's a simpler way to do it, probably so, but this seems dead clear and doesn't potentially leak memory like the previous implementation did (AFAICT). It also means that the entire machinery of cleanup can be handled from the front by just close alone. Having separate user facing methods for close and cleanup was IMO unnecessary baggage.

Also included are tests for SerialPort. Unfortunately, I tried but was unable to find a programmatic way to run these tests on Windows; I would appreciate some manual testing from Windows users!

It would also be delightful to know if this compiles on RPi.

Quick progress checklist:

  • Discuss modernization/changes to interface
  • Test on Windows
  • Check on RPi

timblechmann and others added 30 commits June 23, 2018 21:20
Signed-off-by: Tim Blechmann <tim@klingt.org>
rethrow any ctor exceptions to get better messages
test pointers directly; remove redundant casts
…throwing

this matches the language level interface
deprecate timeout argument, return boolean for put
- change the way callbacks work at end of life
- deprecate cleanupAll
- use dedicated isOpen variable instead of dataptr introspection
- add TODOs & comments
@mossheim
Copy link
Contributor Author

OK, this is ready for testing and review. I have updated the top comment with info on the changes.

@mossheim
Copy link
Contributor Author

mossheim commented Jun 30, 2018

Has anyone else had a chance to look at this yet? @scztt and I walked through the logic together and found small issues with GC and state that I've taken care of in the last few commits. It would be nice if someone could try hooking it up to a pre-existing system (Arduino, etc.) to do some real-world sanity testing.

@nhthn nhthn self-assigned this Jul 9, 2018
@nhthn
Copy link
Contributor

nhthn commented Jul 9, 2018

have verified the tests pass on Linux. going to get a Windows build up this week and test out with an arduino.

@nhthn nhthn changed the base branch from develop to 3.10 July 15, 2018 19:08
@nhthn
Copy link
Contributor

nhthn commented Jul 15, 2018

okay i am STILL getting set up on windows because i am a dumb ass of life, but i have gotten a chance to test reading and writing bytes on linux with an actual arduino uno.

serialtest.ino:

void setup() {
  Serial.begin(9600);
}

int incomingByte = 0;
void loop() {
  if (Serial.available() <= 0) {
    return;
  }
  incomingByte = Serial.read();
  Serial.print("I received: ");
  Serial.println(incomingByte, DEC);
}

supercollider code:

(
var port;
port = SerialPort("/dev/ttyACM0", baudrate: 9600);

Routine.run {
    var character, line;
    loop {
        character = port.read.asAscii;
        (character == $\n).if {
            line.notNil.if { line.join("").postln };
            line = List[];
        } {
            line.add(character);
        };
    };
};

Routine.run {
    var i = 0;
    loop {
        port.put(i);
        i = i + 1;
        1.wait;
    };
};
)

i haven't tested every implemented feature, but this is good news -- seems to be working great.

@nhthn
Copy link
Contributor

nhthn commented Jul 24, 2018

update: got it working on windows!

one bug is that we don't have SerialPort.devices on windows. i am digging into how the arduino IDE discovers serial ports. here is the arduino IDE serial port discoverer, and here is the windows-specific code it uses in the JSSC library.

EDIT: the latter link tells us that we can get a list of devices, identical to that of the arduino IDE, by running

reg query HKLM\HARDWARE\DEVICEMAP\SERIALCOMM\

so all we have to do is parse this.

Nathan Ho added 2 commits July 23, 2018 19:41
This commit adds a working implementation of SerialPort.devices for
Windows by inspecting the HKLM\HARDWARE\DEVICEMAP\SERIALCOMM registry
key.
This commit makes the following changes to SerialPort.devices:

- Add more rules for matching device patterns, imported from the Arduino
  IDE.
- Document SerialPort.devicePattern as a legacy feature. It makes two
  wrong assumptions:
  - Devices can be effectively matched using simple
    file globbing (Linux/macOS require regexps, and Windows requires
    running a command and parsing its output).
  - SerialPort.devices actually needs this kind of customization. It's
    not hard at all to get finer control by post-processing its output
    or just rewriting this simple method.
- Emit a deprecation warning if a number is passed for the "port"
  argument to SerialPort. Brittle and unnecessary convenience.
@nhthn
Copy link
Contributor

nhthn commented Jul 24, 2018

latest commit has a minor API change -- i improved the matching rules for SerialPort.devices for macOS and linux so they're (as far as i can tell) identical to the rules used in arduino. i marked SerialPort.devicePattern as legacy for reasons explained in the docs and my commit messages. i also deprecated passing an integer to the port argument of SerialPort.new, which is a really brittle and unnecessary convenience.

This commit disables one non-functioning SerialPort test, and skips
SerialPort tests if the OS is Windows or it is determined that the
socat utility is not installed.

There is currently no skip mechanism in UnitTest, and due to time
constraints we can't implement it right now. So I've just hacked it
together using a guard clause.
@nhthn
Copy link
Contributor

nhthn commented Aug 3, 2018

in the interest of pushing this along, i have commented out a nonfunctioning unit test and have used guard clauses to skip the tests if we're on Windows or if the "socat" utility is not installed.

@mossheim
Copy link
Contributor Author

mossheim commented Aug 7, 2018

Updated the tests, I believe based on the discussion at the dev meeting that this should be good to go now.

Copy link
Contributor

@nhthn nhthn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks!

@nhthn nhthn merged commit 8b0deb4 into 3.10 Aug 7, 2018
@mossheim mossheim deleted the topic/serialport branch August 7, 2018 21:30
@sensestage
Copy link
Contributor

Just saw that this is going on, great work!

Some remarks (without having tested things yet; I'm quite out of sync with the latest SC developments):

  • devicePattern: this is quite useful, as e.g. on Linux you can also look at /dev/serial/by-id/* instead of /dev/ttyUSB* with the advantage that the first uses fixed names based on the actual device, and the ttyUSB ones are in order of plugging in. (For installations that need to run unattended, I usually use the first one).
  • was a test done where the device gets unplugged? This is what I originally implemented the doneAction for, for accidental device removal or similar, so that you can assign an action to catch that issue (like starting a task that will look for the device to come back).

@mossheim
Copy link
Contributor Author

mossheim commented Sep 3, 2018

Just saw that this is going on, great work!

Some remarks (without having tested things yet; I'm quite out of sync with the latest SC developments):

  • devicePattern: this is quite useful, as e.g. on Linux you can also look at /dev/serial/by-id/* instead of /dev/ttyUSB* with the advantage that the first uses fixed names based on the actual device, and the ttyUSB ones are in order of plugging in. (For installations that need to run unattended, I usually use the first one).

I'm not sure if there is a question here?

  • was a test done where the device gets unplugged? This is what I originally implemented the doneAction for, for accidental device removal or similar, so that you can assign an action to catch that issue (like starting a task that will look for the device to come back).

Yes, that is what test_connectionLost does.

@nhthn
Copy link
Contributor

nhthn commented Sep 4, 2018

devicePattern: this is quite useful, as e.g. on Linux you can also look at /dev/serial/by-id/* instead of /dev/ttyUSB* with the advantage that the first uses fixed names based on the actual device, and the ttyUSB ones are in order of plugging in. (For installations that need to run unattended, I usually use the first one).

thanks for looking at the PR. i changed the device matching behavior, so i should be the one to justify it.

SerialPort.devices was just a thin wrapper around pathMatch. a minor problem is that this thin wrapper isn't necessary: you can just use pathMatch if you want a custom path matching operation.

a major problem is that pathMatch isn't adequate to actually match all the devices we want on all platforms. for macOS and linux, a full-on regexp is needed (pathMatch isn't a regexp, it's a file glob, so there is no way to match e.g.(ttyS|ttyUSB|ttyACM|ttyAMA|rfcomm|ttyO)[0-9]{1,3}). for Windows, we need to make a system call and read the windows registry to figure out what serial ports we have, which is definitely out of the file globbing league.

the serial port primitives aren't coupled to the device matching behavior. SerialPort.new doesn't care if the device you pass in wouldn't be matched by one of the built-in heuristics. so if you need a special file glob, you can just run "/dev/serial/by-id/*".pathMatch, pick a device name, and pass it into SerialPort.new.

due to the complete decoupling of device matching and serialport creation, there really isn't any need for a user to customize SerialPort.devices -- the user should just swap it out for their own path matching and SerialPort.new will be fine with it. hope that makes sense.

@sensestage
Copy link
Contributor

ok. that makes sense.
Thanks for the explanation!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
API change comp: sclang sclang C++ implementation (primitives, etc.). for changes to class lib use "comp: class library" waiting for testing
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants