Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dictionary == operator produces unexpected behavior #2737

Closed
firsttempora opened this issue Feb 21, 2017 · 17 comments
Closed

Dictionary == operator produces unexpected behavior #2737

firsttempora opened this issue Feb 21, 2017 · 17 comments
Labels
API change comp: sclang sclang C++ implementation (primitives, etc.). for changes to class lib use "comp: class library"

Comments

@firsttempora
Copy link

firsttempora commented Feb 21, 2017

There seems to be an issue with the == operator with dictionaries: it only considers values (not key associations) for testing equality. Some MWEs that demonstrate the problem:

a = Dictionary.newFrom([\a, 1]);
b = Dictionary.newFrom([\b, 1]);
a == b // Returns true
a = Dictionary.newFrom([\a, 1, \b, 2, \c, 2]);
b = Dictionary.newFrom([\a, 1, \b, 2, \c, 3]);
a == b; // evaluates to true
// Order also matters (which it shouldn't):
b == a; // evaluates to false

It seems that Dictionary inherits the == operator from Collection, which relies on Dictionary's implementation of includes, so whether it is appropriate to fix includes or override == in Dictionary seems to me to be an open question. It's not immediately obvious what includes should do - whether it operates on keys, values, or associations (i.e. a method includesValue would clearly return true if the given value occurs in the dictionary, no matter what key it is associated with, but includes is more ambiguous.)

There was some discussion of the appropriate behavior of set theory methods for dictionaries here but no discussion of the == operator there.

@firsttempora
Copy link
Author

firsttempora commented Feb 21, 2017

One possible solution we've found is to override the Dictionary == method as such:

+ Dictionary{
    == {
        arg otherDict;
        ^(this.asSortedArray == otherDict.asSortedArray);
    }   
}

This produces the expected behavior; there may be better implementations.

@mossheim
Copy link
Contributor

Interesting. This is worth some discussion, and would be a significant API change, but I don't have any opinion yet. The only thing I'll say for now is that I would be happy to write a C++ primitive for any new method, because whatever SC implementation would be used would undoubtedly be very slow.

@mossheim mossheim added API change comp: sclang sclang C++ implementation (primitives, etc.). for changes to class lib use "comp: class library" labels Feb 21, 2017
@vivid-synth
Copy link
Member

Ouch ouch ouch!

Discussion of API changes usually happens on the mailing list ( https://supercollider.github.io/community/mailing-lists.html ) -- maybe we can discuss the changes there?

@firsttempora
Copy link
Author

Should I get the ball rolling over on the SC mailing list (sc-devs I'm assuming)?

@telephon
Copy link
Member

The case is pretty uncontroversial, I'd expect, even if it is a breaking API change. If anyone has used == on dictionaries, she will have been surprised anyhow.

The implementation, however, is something to debate. (think parent and proto envirs etc.)
I'd suggest a "behavioral" definition a bit like:

== { |that|
	if(that.isKindOf(this.class).not) { ^false };
	this.keysValuesDo { |key, val|
		if(that.at(key) != val) { ^false }
	};
	that.keysValuesDo { |key, val|
		if(this.at(key) != val) { ^false }
	};
	^true

}

this can be slow in some circumstances, so an extra primitive is a good idea.

@ekermit
Copy link

ekermit commented Mar 1, 2017

I would suggest checking the number of entries too before iterating. Also, I don't think there is a reason to iterate over both dictionary key-sets.

== { |that|
	if(that.isKindOf(this.class).not) { ^false };
	if(that.keys.size != this.keys.size) { ^false };
	that.keysValuesDo { |key, val|
		if(this.at(key) != val) { ^false }
	};
	^true
}

@mossheim
Copy link
Contributor

mossheim commented Mar 1, 2017

The implementation is trivial, and it should be in C++. Any SC implementation is likely to be an order of magnitude slower.

Note that changing this would affect all of Dictionary's subclasses:

  • IdentityDictionary
  • Environment
  • NodeMap
  • Event
  • ProxyNodeMap

I've emailed the sc-dev list :)

@telephon
Copy link
Member

telephon commented Mar 1, 2017

once the implementation for Dictionary is done, I'll add the necessary additions to the subclasses. Note that probably IdentityDictionary needs an extra branch in the primitive, because it has proto and parent instvars.

@telephon
Copy link
Member

telephon commented Mar 1, 2017

one more thing: anything that implements == should also implement hash. This is e.g. necessary for it to work in Set, or as keys of Dictionary.

@jamshark70
Copy link
Contributor

I would suggest checking the number of entries too before iterating. Also, I don't think there is a reason to iterate over both dictionary key-sets.

I puzzled over this for a minute, but it turns out it's a little deeper than that: checking the size appears to eliminate the need to iterate over both sides!

That is, the potential failure case with looping over only one side is if the other side contains an entry not present in the side you looped through -- but in that case, the sizes will be different.

@telephon
Copy link
Member

telephon commented Mar 2, 2017

@ekermit the message .keys will generate a new object (a Set of keys), so it's not optimal. Better to check directly:

== { |that|
	if(that.isKindOf(this.class).not) { ^false };
	if(that.size != this.size) { ^false };
	that.keysValuesDo { |key, val|
		if(this.at(key) != val) { ^false }
	};
	^true
}

For IdentityDictionary, we'll need:

== { |that|
	^this.superPerform('==', that) 
		and: { parent == that.parent } 
		and: { proto == that.proto } 
		and: { know == that.know }
}

@eirikblekesaune
Copy link
Member

I was about to report an issue with equality for Sets with objects that have a custom '==' method defined and I found this thread. The proposed solution ( #3011 ) seems to target Dictionary and its subclasses(?), whereas my issue may point to a bug in Dictionary's superclass - Set.

It might be that I going about this the wrong way in general, in which case I hope to be enlightened. Not sure if I should make this a separate issue, so I adding it to this thread. Please let me know if you want me to make it a separate issue.

I am getting unstable results comparing two Set objects containing objects of a class that defines its own equality '==' method. The results returns seemingly random true and false values.

Here is a class I use for testing:

MyClass{
	var <aa;
	var <bb;

	*new{arg aa, bb;
		^super.newCopyArgs(aa, bb);
	}

	=={arg val;
		if(val.isKindOf(this.class).not, {
			^false;
		});
		if((aa == val.aa).not, {
			^false;
		});
		if((bb == val.bb).not, {
			^false;
		});
		^true;
	}

}
// returns steady 'true' values
MyClass(11, 22) == MyClass(11,22); 
//returns steady 'true' values
[MyClass(11, 22)] == [MyClass(11,22)]; 
// returns 'true' and 'false' values arbitrarly
Set[MyClass(11, 22)] == Set[MyClass(11,22)]; 

@mossheim
Copy link
Contributor

mossheim commented Jul 4, 2017

@blacksound Your object doesn't provide a hash function so its hash (inherited from Object's default implementation) is based on identity, not equality.

Note from the docs (Object:-hash):

.hash: Answer a code used to index into a hash table. This is used by Dictionary and Set and their subclasses to implement fast object lookup. Objects which are equal == should have the same hash values. Whenever == is overridden in a class, hash should be overridden as well.

(There's a similar note on == as well)

@eirikblekesaune
Copy link
Member

eirikblekesaune commented Jul 4, 2017

Thanks Brian,
Would thai be a possible solution for hash method?:

hash{
    ^this.instVarHash([\aa, \bb]);
}

@telephon
Copy link
Member

telephon commented Jul 4, 2017

yes, that is the standard way to do it. Recently I noticed that the order should be encoded in instVarHash, too, but that is another issue.

@mossheim
Copy link
Contributor

mossheim commented Jul 4, 2017

@blacksound Yes, in fact you could simplify it to ^this.instVarHash; without arguments, it operates on all the instance variables. if you intend to make your == always check whether all instances variables are equal between instances, this is the clearer route. but maybe you have a cc that can be true or false and MyClass(0, 1, cc: true) == MyClass(0, 1, cc: false). Then you wouldn't want the hash function to hash cc, and you should explicitly list the instance variables you want to check. you'll have to think ahead to decide which of these is better; either option could be viewed as defensive programming, but the best solution is to have a test suite to ensure both equality and hash-equality (that a == b => a.hash == b.hash) for a relevant set of conditions.

@telephon
Copy link
Member

telephon commented Jul 4, 2017

… for equality (==), you have the corresponding compareObject method.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
API change comp: sclang sclang C++ implementation (primitives, etc.). for changes to class lib use "comp: class library"
Projects
None yet
Development

No branches or pull requests

7 participants