Monday, December 22, 2014

Stack Trace Error in Google Sets Intersection

Found a very puzzling bug in some code. I still do not quite understand WHY my fix, fixed it.
Essentially I was sometimes getting a stack trace exception

at com.google.common.collect.Sets$2.contains(Sets.java:655)
at com.google.common.collect.Sets$2.contains(Sets.java:655)

The offending code looked like this (NOT THE ACTUAL CODE)

for (SomeObject m : this) {
  commonTypes = Sets.intersection(commonTypes, m.getSet());
}


This would cause problems, it seemed to be related to this loop being executed a lot. Over 10 000 times, though the set sizes where never greater than 20 or so. Anyway this code fixed it.

for (SomeObject m : this) {
  Set<AttrType> tempTypes = Sets.intersection(commonTypes, m.getSet());
  commonTypes = new HashSet<>();
  commonTypes.addAll(tempTypes );
}


Thursday, December 18, 2014

Using GridFS with MongoJack


We use MongoJack to convert our POJO (plain java) classes to JSON so they can be saved easily into MongoDB. MongoJack also handles the reading of the BJSON from the MongoDB and creating the POJO again. MongoJack internally uses Jackson.

So to persist a java file we can just use the annotations
    @JsonCreator
    @JsonProperty


Then we save to the MongoDB with the JacksonDBCollection.
Our problem was that part of out POJO object was a byte[] that was over 16 Meg. MongoDB has a restriction on document size. So our POJO had a
    byte[] _content;

Massive documents need to be persisted with GridFS in MongoDB. My approach was to leave the Java object with _content but save that part to MongoDB using GridFS. Then add contentId to our POJO to point to the ID of the GridFS saved content.

Steps
1. Stop MongoJack from persisting _content was easy. Just add this annotation to the POJO
    @JsonIgnore
2. On insert save GridFS file and link to POJO with contentId
3. On load/get merely get the contentId and load the appropriated GridFS saved file and read it back into the POJO.

This method replaces the way that the POJO is saved, but leaves the POJO as is. The code does not care how the POJO is saved.




Tuesday, December 16, 2014

Learning MongoDB

Probably the best resource I have found for learning MongoDB quickly is The little MongoDB Book
Very much worth it to go through the book and it only takes an afternoon.

One thing that I immediately liked was the format of the find in the shell.
db.hits.update({page: 'unicorns'},{$inc: {hits: 1}}, {upsert:true});
It just reminds me of groovy and passing and returning json. I compare that to something that might be written in traditional java, which seems so clumsy.
       
           db.getHits().updateWithInc(String page, Integer inc, boolean upsert);

Where page, being a string we would have to be able to parse the string. Or pass an object. With groovy of course any object can just be flattened into JSON and rebuilt.

Far more flexibility, far simpler syntax.