On Mac OS 10.8, using the latest master, I can't reproduce this behaviour. when testing, I used top -pid <process-id-of-sclang>.

I noticed that when I run this bit of unrelated code:
fork { loop { 0.01.wait; 1000.rand } } I get a buildup of CPU till almost 80 % that falls again, in a cycle of 10 seconds or so.

On Mac OS 10.8, using the latest master, I can't reproduce this behaviour.

How long are your code files? The problem does not manifest with short files.

Did some benchmarking -- the culprit is chars = String.fill(chars.size, {|i| chars[i].asAscii})

Usually this is very quick, but it seems quite time consuming for long files (here, I'm omitting the short files that took less than 0.1 sec):

>> syncFromIDE: b-vs-d.scd
chars asAscii -- time to run: 0.23945093154907 seconds.
>> syncFromIDE: mel3-4.scd
chars asAscii -- time to run: 0.16929697990417 seconds.
>> syncFromIDE: ScIDE.sc
chars asAscii -- time to run: 0.16332912445068 seconds.
>> syncFromIDE: 04-defs.scd
chars asAscii -- time to run: 2.1364738941193 seconds.
>> syncFromIDE: perc-92bpm.scd
chars asAscii -- time to run: 1.0300500392914 seconds.

The worst of them, 04-defs.scd, contains 59314 characters. So let's assume syncFromIDE receives an Int8Array.

x = Array.fill(59314, { rrand(32, 127) });
bench { y = String.fill(x.size, { |i| x[i].asAscii }) };
--> time to run: 2.9793999195099 seconds.

Ouch.

bench { y = String.fill(59314, $ ) };
--> time to run: 2.9324381351471 seconds.

Ouch.

That's as far as I can go with it this morning...

While I was out for lunch, I started to suspect that maybe String.fill uses concatenation. Indeed...

    *fill { | size, function | // any output for function is accepted
        var obj = "";
        size.do { | i | obj = obj ++ function.value(i); };
        ^obj
    }

That makes sense, if we aren't assuming that every item will be a Char.

If I rewrite syncFromIDE as follows:

    *syncFromIDE {|quuid, title, chars, isEdited|
        var doc, argChars;
        isEdited = isEdited.booleanValue;

        // this bit is new
        argChars = chars;
        chars = String.new(argChars.size);
        argChars.do { |ch| chars.add(ch.asAscii) };

        if((doc = this.findByQUuid(quuid)).isNil, {
            doc = super.prBasicNew.initFromIDE(quuid, title, chars, isEdited);
            allDocuments = allDocuments.add(doc);
        }, { doc.initFromIDE(quuid, title, chars, isEdited) });
    }

... then 2.1364738941193 seconds to process 04-defs.scd in my earlier test turns into 0.012862205505371, some 166 times faster.

I suppose it would be useful to add a *fillChars method to String? So that people would be less tempted to use the general-case but massively slower String.fill?

Oh..... wslib overwrites String *fill with the slower version. So that's why nobody else could reproduce it.

(Insert boilerplate lecture about overwriting core methods...)

So I guess the best fix is the above, since quarks will not be able to interfere with it then (unless some...one decides that syncFromIDE would be a good candidate for "improvement"...).

I think the best solution would be to have Wouter (or somebody else) switch that to a new method in wslib.

Closing for now.

ok, fixed in wslib. String:fill is now renamed to String:fillJoin (and String:collect to String:collectJoin)

Thanks!

On 3 Nov 2013, at 12:20, Wouter Snoei notifications@github.com wrote:

ok, fixed in wslib. String:fill is now renamed to String:fillJoin (and String:collect to String:collectJoin)

—
Reply to this email directly or view it on GitHub.