Stack Trace Error in Google Sets Intersection

Found a very puzzling bug in some code. I still do not quite understand WHY my fix, fixed it.
Essentially I was sometimes getting a stack trace exception

at com.google.common.collect.Sets$2.contains(Sets.java:655)
at com.google.common.collect.Sets$2.contains(Sets.java:655)

The offending code looked like this (NOT THE ACTUAL CODE)

for (SomeObject m : this) {
  commonTypes = Sets.intersection(commonTypes, m.getSet());
}


This would cause problems, it seemed to be related to this loop being executed a lot. Over 10 000 times, though the set sizes where never greater than 20 or so. Anyway this code fixed it.

for (SomeObject m : this) {
  Set<AttrType> tempTypes = Sets.intersection(commonTypes, m.getSet());
  commonTypes = new HashSet<>();
  commonTypes.addAll(tempTypes );
}