JavaOne Shanghai

I was expecting this...

So, I was in China last week.  It's not really what I expected, but then my knowledge of China comes almost entirely from visiting various Chinatowns and watching martial arts movies, so I guess I wasn't all that well prepared for a modern Asian city like Shanghai.

...and found this instead

The city is, in my opinion, clean, modern, and attractive.  I was surprised by American-looking early skyscrapers, and really surprised that the city overall felt more European than American - with Marks & Spencer, Tesco, Carrefour, H & M, Zara, Costa Coffee, as well as the inevitable McDonalds, KFC and Pizza Hut.

Some of the architecture reminded me more of New York than Europe or Asia

I felt like they had the option to take the best of all worlds - the modern, high tech and sometimes downright weird influence of Japan; a care for heritage from Europe; the consumer choice of the US; and food from all over China and the world.

OK, I didn't eat here, but I did love the food in Shanghai

The food was awesome and, by my London-standards, very reasonably priced.  Our hotel was amazing, and also a good price for such luxury (although it was in the arse end of nowhere, but that didn't matter when taxi journeys to the centre of Shanghai were around £3.00).  Road users were mental (although there are probably many worse places I haven't visited them yet) and we got into the habit of not looking out of the cab windows when we travelled.

Apparently road accidents kill 300 people a day in China.

I did miss Twitter and Facebook while I was out there, and the WiFi availability was limited.  Also, maybe I'm paranoid, but I wasn't sure how much to trust the WiFi.  But I found myself quite pleased to be separated from the interwebs for a bit - it was peaceful and relaxing.  I even did more work than I expected, probably because I wasn't replying to e-mails all the time or thinking of what to tweet next.

The conference itself was a weird mix of the familiar with the massively unfamiliar - familiar, because it felt like a JavaOne - the branding, the badges, the signage.  Unfamiliar because I don't speak Chinese - most of the signs and documentation were in Chinese, and not all the stewards spoke English.  The attendees in particular were not expected to speak English (which is totally understandable of course! Conferences should cater for the local population), which made it more difficult to feel like a part of the event.  I think I've mentioned before that the more I come to these events, the less I feel like an outsider as one of the rare women.  But as a blonde-haired European woman I did feel a little... conspicuous.

It was an Oracle conference.  In case you're not sure.

Oracle probably like the fact that the event is small enough that Open World and JavaOne can be in the same place at the same time, but to me it feels a bit odd.  I couldn't easily find the content that was applicable to me as a Java developer, and I really don't see much crossover between the two conferences - I can't see what I, as a developer, would get out of "How to use Oracle blah blah more effectively". And I can't see the suited business types enjoying "Effective Scala".

It's less obvious than at the SF conference(s), but there's a very distinct difference between the JavaOne attendees (emphasis on the casual part of business casual) and the Oracle people (do they sleep in their suits?). I didn't envy the latter at all - wearing a full suit and tie in 40°C heat and humidity you could drown in must be extremely uncomfortable. Speaking of clothes, all the women are dressed in summery, pretty clothes (except the business-suited Oracle types), and looked like normal human beings, and the geek guys were wearing less badges of geekness (geek t-shirts, jeans etc).

This did seem to be a great conference to get people from all over Asia and even the antipodes together. There was buzz in the air, an excitement to be learning stuff and meeting people.  I don't know if it's the (relatively) small size of the conference, its youth or its geography, or maybe simply me projecting my excitement at being in a place I'd never been to before, but there was less cynicism than in the western conferences.

Awesome to see the LJC get a shout out in China!

In terms of presenting, this was an entirely new experience for me.  The slides had all been translated into Chinese¹, which I've already mentioned I don't speak, so this meant I had to have my laptop out with my English notes on, as well as the slides on the main screen.  I had two remotes, one in each hand, and I advanced both sets of slides at the same time. The fact that the slide on the screen was also on a monitor in front of me made it relatively easy to make sure I had both in sync, but it did require a bit of extra cognitive power.  I felt lucky that most of my slides are practically content-less - if my presentations followed the Bullet-Riddled Corpse pattern a) the poor translator would have had to work harder and b) it would have been more difficult to work out which slide I was on.

Half of my massive audience

In addition, my presentation was being interpreted in real time as I spoke.  I was worried about the interpreter - I know I speak too fast, and I wasn't sure if Chinese is one of those languages where it takes more words to say the same thing².  So I needed to make sure I slowed down, spoke clearly and coherently, and left longer pauses.  These are things you should do as a presenter anyway, but it always feels a little unnatural.  And in fact the more you present, the easier it is to forget this, especially if you get good feedback on your talks.

To deal with both these different things (Chinese slides and real time interpretation), I had to be prepared earlier than normal and I practiced more than normal.  I wanted to get the timing right so I would be comfortable slowing down.  I wanted it to be 40 minutes (in a 60 minute slot) to give me 10 mins for questions and 10 mins buffer for allowing me to slow down and/or cope with any disasters.  I needed to run through with the Chinese slides so there weren't any nasty surprises.

In the end, I thought it went very well.  The preparation made me feel much more comfortable and confident.  Slowing down had an additional benefit - not everyone opted for the headphones & interpreter, so me not speaking at 100 miles an hour gave them a fighting chance of following along.  The audience seemed very reluctant to ask questions in public, so I skipped my usual audience-participation bit, but the fact that I hit 40 minutes exactly gave people loads of time to come up to me afterwards and ask me questions, and I did have quite a big line of people as I was packing up. I think this is a Good Thing.

Fortunately (for me) the session was not recorded in any form, so you won't be able to see whether it really was any good or not.  But here are the slides, in case you speak Chinese (sadly my Keynote to Powerpoint conversion didn't quite work, so some of the code falls off the bottom).

(Note that my caveats are in Chinese, but the caveats are that the new API for the MongoDB Java driver will almost definitely NOT look like that, it's just indicative of the direction we were looking in when I wrote that presentation.)

Although I was more nervous about this session than recent sessions, I'm really pleased I did it.  I highly recommend this to other speakers as a way to force you out of your routine and to think about your presentation style (something we probably all stress about anyway, but this forces you to look carefully at it).


And Shanghai overall... as you can probably gather from the start of this blog, I liked it a lot.  I had a nice, mostly relaxing, fun time there.  I loved the modern-ness of it, like the ridiculously fast MagLev that takes a mere 7 minutes to travel the 30km to the airport.

The Shanghai MagLev is "the world's fastest train in regular commercial service"

I liked the alienness, after countless European and American cities.  I did not like not being able to speak the language, or the fact that taxi drivers don't want to take you to the Expo area where our hotel was.  I liked the many touches of familiarity and the atmosphere.  I'd definitely go back.

Pollution makes awesome sunsets

And it's whetted my appetite for JavaOne so I'm really looking forward to San Francisco in September. See you there?

¹ I was surprised to hear everyone talking about "Chinese" when even in my ignorance I know there are many types of Chinese language, but doing my research I've learnt there is a Standard Chinese.  Which makes sense.  You learn something new every day
² Apparently it's not

Can't. Take. Much. More.

Inspired by Your App Makes Me Fat, I was going to write a blog post about decision fatigue.  It's a topic very close to my heart right now as I try to plan for my upcoming speaking engagements, continue working on the new Java driver, try to finish the last couple of blogs on Spock, and last but not least plan for moving home this week.

But since my cognitive resources are at a low point right now, I'll just leave you with a gif instead:

What programming is like

Spock passes the next test - Painless Stubbing

In the last post I talked about our need for some improved testing tools, our choice of Spock as something to spike, and how mocking looks in Spock.

As that blog got rather long, I saved the next installment for a separate post.

Today I want to look at stubbing.

Stubbing
Mocking is great for checking outputs - in the example in the last post, we're checking that the process of encoding an array calls the right things on the way out, if you like - that the right stuff gets poked onto the bsonWriter.

Stubbing is great for faking your inputs (I don't know why this difference never occurred to me before, but Colin's talk at Devoxx UK made this really clear to me).

One of the things we need to do in the compatibility layer of the new driver is to wrap all the new style Exceptions that can be thrown by the new architecture layer and turn them into old-style Exceptions, for backwards compatibility purposes.  Sometimes testing the exceptional cases is... challenging.  So I opted to do this with Spock.

So here we can use a real DB class, but with a mock Mongo that will return us a "mock" Session.  It's not actually a mock though, it's more of a stub because we want to tell it how to behave when it's called - in this test, we want to force it to throw an org.mongodb.MongoException whenever execute is called.  It doesn't matter to us what get passed in to the execute method (that's what the underscore means on line 16), what matters is that when it gets called it throws the correct type of Exception.

Like before, the when: section shows the bit we're actually trying to test. In this case, we want to call rename.

Then finally the then: section asserts that we received the correct sort of Exception.  It's not enormously clear, although I've kept the full namespace in to try and clarify, but the aim is that any org.mongodb.MongoException that gets thrown by the new architecture gets turned into the appropriate com.mongodb.MongoException.  We're sort of "lucky" because the old code is in the wrong package structure, and in the new architecture we've got a chance to fix this and put stuff into the right place.

Once I'd tracked down all the places Exceptions can escape and started writing these sorts of tests to exercise those code paths, not only did I feel more secure that we wouldn't break backwards compatibility by leaking the wrong Exceptions, but we also found our test coverage went up - and more importantly, in the unhappy paths, which are often harder to test.

I mentioned in the last post that we already did some simple stubbing to help us test the data driver. Why not just keep using that approach?

Well, these stubs end up looking like this:

Ick.

And you end up extending them so you can just override the method you're interested in (particularly in the case of forcing a method to throw an exception).  Most irritatingly to me, these stubs live away from the actual tests, so you can't easily see what the expected behaviour is.  In the Spock test, the expected stubbed behaviour is defined on line 16, the call that will provoke it is on line 19 and the code that checks the expectation is on line 22.  It's all within even the smallest monitor's window.

So stubbing in Spock is painless.  Next:

• Data Driven Testing

Spock is awesome! Seriously Simplified Mocking

We're constantly fighting a battle when developing the new MongoDB Java driver between using tools that will do heavy lifting for us and minimising the dependencies a user has to download in order to use our driver.  Ideally, we want the number of dependencies to be zero.

This is not going to be the case when it comes to testing, however.  At the very least, we're going to use JUnit or TestNG (we used testng in the previous version, we've switched to JUnit for 3.0).  Up until recently, we worked hard to eliminate the need for a mocking framework - the driver is not a large application with interacting services, most stuff can be tested either as an integration test or with very simple stubs.

Recently I was working on the serialisation layer - we're making quite big changes to the model for encoding and decoding between BSON and Java, we're hoping this will simplify our lives but also make things a lot easier for the ODMs (Object-Document Mappers) and third party libraries.  At this level, it makes a lot of sense to introduce mocks - I want to ensure particular methods are called on the writer, for example, I don't want to check actual byte values, that's not going to be very helpful for documentation (although there is a level where that is a sensible thing to do).

We started using JMock to begin with, it's what I've been using for a while, and it gave us what we wanted - a simple mocking framework (I tried Mockito too, but I'm not so used to the failure messages, so I found it really hard to figure out what was wrong when a test failed).

I knew from my spies at LMAX that there's some Groovy test framework called Spock that is awesome, apparently, but I immediately discarded it - I feel very strongly that tests are documentation, and since the users of the Java driver are largely Java developers, I felt like introducing tests in a different language was an added complexity we didn't need.

Then I went to GeeCON, and my ex-colleague Israel forced me to go to the talk on Spock.  And I realised just how wrong I had been.  Far from adding complexity, here was a lovely, descriptive way of writing tests.  It's flexible, and yet structured enough get you thinking in a way that should create good tests.

Since we're already using gradle, which is Groovy as well, we decided it was worth a spike to see if Spock would give us any benefits.

During the spike I converted a selection of our tests to Spock tests to see what it looks like on a real codebase.  I had very specific things I wanted to try out:

• Mocking
• Stubbing
• Data driven testing

In the talk I also saw useful annotation like @Requires, which I'm pretty sure we're going to use, but I don't think it's made it into a build yet.

So, get this, I'm going to write a blog post with Actual Code in.  Yeah, I know, you all thought I was just a poncy evangelist these days and didn't do any real coding any more.

First up, Mocking
So, as I said, I have a number of tests checking that encoding of Java objects works the way we expect.   The easiest way to test this is to mock our BSONWriter class to ensure that the right interactions are happening against it.  This is a nice way to check that when you give an encoder a particular set of data, it gets serialised in the way BSON expects. These tests ended up looking something like this:

(Yeah, I'm still learning Spanish).

So that's quite nice, my test checks that given a List of Strings, they get serialised correctly.  What's not great is some of the setup overhead:

Obviously some of the things there are going to be ringing some people's alarm bells, but let's assume for a minute that all decisions were taken carefully and that pros and cons were weighed accordingly.

So:

• Mocking concrete classes is not pretty in JMock, just look at that setUp method.
• We're using the JUnitRuleMockery, which appears to be Best Practice (and means you're less likely to forget the @RunWith(JMock.class) annotation), but checkstyle hates it - Public Fields Are Bad as we all know.

But it's fine, a small amount of boilerplate for all our tests that involve mocking is an OK price to pay to have some nice tests.

I converted this test to a Spock test.  Groovy purists will notice that it's still very Java-y, and that's intentional - I want these tests, at least at this stage while we're getting used to it, to be familiar to Java programmers, our main audience.

Some initial observations:

• It's a really simple thing, but I like having the @Subject annotation on the thing you're testing.  In theory it should be obvious which of your fields or variables is the subject under test, but in practice that's not always true.
• Although it freaks me out as someone who's been doing Java for the last 15 years, I really like the String for method name - although in this case it's the same as the JMock/JUnit equivalent, it gives a lot more flexibility for describing the purpose of this test.

• Mocking is painless, with a simple call to Mock(), even though we're still mocking concrete classes (this is done simply by adding cglib and obgenesis to the dependencies).
• I love that the phases of Spock (setup: when: then:) document the different parts of the test while also being the useful magic keywords which tell Spock how to run the test.  I know other frameworks provide this, but we've been working with JUnit and I've been in the habit of commenting my steps with //given //when //then.
• Thanks to Groovy, creation of lists is less boiler plate (line 9).  Not a big deal, but just makes it easier to read.
• I've got very used to the way expectations are set up in JMock, but I have to say that 1 * bsonWriter.blahblahblah() is much more readable.  
• I love that everything after then: is an assertion, I think it makes it really clear what you expect to happen after you invoke the thing you're testing.

So mocking is awesome.  What's next?

• Painless Stubbing
• Data Driven Testing

Developers hate support, don't they?

This is Dublin.  Honest.

I'm at the end of my first official week doing support for 10gen.  My major achievements are:

• Learning how to work the coffee machine in the Dublin office.  It's taken me a week to get it, but now I can understand the machine's needs.  Even if my coffee did taste a bit of cleaning fluid this morning.
• Navigating around Dublin - I led the way to the pub, followed by two people who live here. Turns out I know the most efficient routes between work venues and pubs, even in a town that I've never been to before.
• Closing all outstanding Java driver pull requests, except for the one we want to merge but needs to be updated.  This is a massive win for us, now we can actually stay on top of them instead of wondering which are valid and which ones we're actively ignoring.  I hope the CTO isn't peeved I closed one of his from two years ago.
• I got my highest scoring day on StackOverflow yesterday, receiving a "whopping" 67 points in one day.  Be nice, I'm quite new to SO - despite being on there for over two years I'm really only just starting to feel comfortable there.  One day I might actually ask a question.

In all seriousness though, I learnt a LOT this week.  I've been with 10gen for about 9 months now, so although I feel like I should be a fully fledged member of the team there's still a lot I don't know about.  In particular, as I hadn't used MongoDB before working here, and as I'm head-down on delivering the new Java driver (more to come on that later, I promise), I don't see a lot of how people actually use the database.

I've been lucky because this wasn't a heavy week for customer support, so I've been available to pick up "free" support - basically take a look at questions people are asking on StackOverflow and the MongoDB Google Groups and see if there's anything I can answer.  Obviously I focussed on the Java side as this is the area I'm most comfortable (and there are only a few of us to answer those questions, and plenty of others to pick up more general questions).  

So here are my Lessons Learnt:

Customers want to be heard

Responsiveness is almost more important to (paying) customers than detailed answers.  They want to know that you've seen their problem, and you're on it.  So sometimes it's best to answer a problem with a question or questions, asking for clarification on their situation and explaining any assumptions you have.  It turns out that writing that long and detailed explanation of the steps they need to take isn't really necessary when you've figured out what they're really trying to do and response with "this is going to require production downtime". Suddenly it's not as urgent for them to fix this right now.

The Java Ecosystem is alive and well.  In fact, it's a monster...

There really are a remarkable number of libraries/third party apps that people are using on top of our basic Java driver.  This week, I've been trouble-shooting people using Morphia, Spring Data, Jongo, Grails GORM, as well as a lot of people who are using the driver directly.  I've learnt a lot about installing and using all these things, as I've had to do that to reproduce people's issues and suggest solutions¹.  I've learnt that some of those things make life quite a lot easier, especially for the happy path of Java developers who simply want to save Java pojos into the DB and query for them afterwards.

I've also learnt that querying is quite inconsistent across all the libraries, and I'd say the main problem that users have is turning an understandable query from the shell into something that the library they're using understands.

Also, Dates get people every time, particularly in Java.

Where is the problem?

Related to the previous point, when a query returns nothing it's difficult to work out: if you're using the library incorrectly; if the Java driver doesn't support various options; or if the database isn't doing what you expect it to.  More often than not the problem is not understanding how to use the tools (and the onus is on us, the library developers, to make it as easy and unsurprising as possible for developers), but sometimes it's a bug.  Tracking down the root of the issue is important, because if it's a bug we should log it in the correct place, and if it's working as designed it suggests a lack of documentation or possibly a design flaw.

Having people to help you is dead important

I could have done this from home, from the London office, or  even from the beach if I'd wanted to. However, I opted to come to our Dublin office where we have our EMEA support, because I knew that alone I'd try and dodge the work.  Not because I don't want to do it, but because I don't know our processes, I don't know if we have a knowledge base I can use, I don't know if my answers are correct, and so on and so on.  When you're uncertain, sometimes the easiest option is to run away. But when you're embedded in the team that does this every day, you can ask the stupid questions and get help rather than sit and feel stupid.  

It's also nice to be part of a daily standup so you can hear what other people are up to, and it puts a (good) subtle pressure on you to do something so you have something to report.

What I like about support at 10gen

I've done support before.  In fact, during my last year at LMAX I worked out I spent the same amount of time on the production support team as I did working on a team delivering new features.  But since I've never worked for a product firm before, I haven't done this sort of support.  10gen makes its money by selling support contracts, so not only is it important to keep the SLAs with the paying customers, but it's also important to provide excellent quality support for them - with any luck they'll keep paying.  It's also important for us to support those who don't pay - it's kinda like a teaser of the great quality they'd get if they paid us.

So the support organisation here is absolutely key to our bottom line.

The driver developers (of which I am one) are "strongly encouraged" to do a week of support every three months.  We used to do days here and there, but blocking out a whole week to be part of the team, to be excused from your day job, and to leave your e-mail (and sometimes meetings) alone because you're busy doing support, works really well.  I've felt very focussed on the support role during this week, and I've been motivated to do a good job.  If I had been seconded to the team for a period of weeks, I would have dreaded the whole thing and have plodded along doing my time until my escape. Alternatively, doing it for a day at a time the support issues leak into the day job and the day job is hard to put down during your support hours.

I've learnt a lot this week.  I have been lucky, I've had fairly straightforward issues to address and it hasn't been a busy week.  But next time I see the support week looming in my calendar, I won't fear it as much.  I'll probably do the next one from home, but my week here in Dublin has prepared me for what's in store.  Most importantly, I've got to know the people who are part of my team here, so I'll be more comfortable asking the stupid questions.

In summary

• Doing support is so important for developers.  Getting an understanding of what your users are doing and what their pain points are can really help drive your design and your architecture, and can really motivate you to fix that stupid bug or write that overdue tutorial.
• Developers and support techies can learn off each other - I've learnt a LOT off the guys here, but I've also been able to answer a few questions about Java in general and the driver in particular.
• Scheduling time in support for developers is a very tricky balance, and probably needs to be experimented with a bit before a company finds something that works for them.  

Although I'm never really going to love being on support, I still like being encouraged to work on support regularly - it's been so important for my understanding of our product and of our users.

¹ I found Grails BY FAR the most difficult to get started with - all I wanted was to write a unit test to prove the issue, and I had to not only create a whole application using the grails command line, but I had to install a whole host of plugins and uninstall all the stuff for hibernate that gets put in there magically.  I never did find the solution to the Grails problem I was looking at.