Introduction to the Disruptor Slides

I've produced a very cut down version of the presentation I've been giving at a lot of conferences, giving a high level overview to the Disruptor.  This serves as a quick intro to the concepts behind it.

Introduction to the Disruptor from Trisha Gee

My slides are usually pretty useless without me (or someone else) talking over them, so for more context don't forget there's always my original blog posts (the Magic Ring Buffer, Reading from it, Writing to it, Wiring it up), which are now pretty dated, and the Java Magazine article I wrote at the start of the year.

Strangeloop Disruptor Workshop Materials

On Sunday I gave my very first workshop on the Disruptor.  The aim was to give people some hands-on coding experience using the syntax.  Because time was limited (you can't get people to build an entire application architecture in 2.5 hours) the example is somewhat contrived, and needs a big leap to make it into a proper application context.  But the workshop should:

1. Give an overview of the Disruptor
2. Show how to create a simple one producer, one consumer example.
3. Show how to wire up a parallel event handler
4. Show how to how (and why) to create a diamond dependency graph.
5. Extrapolate beyond these very simple examples to something closer to a real world architecture.

Requirements are:

1. Basic Java skills
2. Java 7 update 7
3. (Optionally) IntelliJ.

The slides (not all that useful, I'll grant you, without me talking) are available, and you can download the code.

If you want to work through the examples yourself, start with com.mechanitis.towerdefense.TowerDefence.  The starting point code for the workshop is in package com.mechanitis.towerdefense.  com.mechanitis.towerdefense1 shows a solution to exercise one, com.mechanitis.towerdefense2 to exercise 2 and so forth.  The final package, com.mechanitis.towerdefense4, has some different towers you can play with to see the effects.  They don't really show anything special about the Disruptor, but they were fun to write and are interesting to play with.

QCon London: Disruptor presentation video

The "User's Guide to the Disruptor" presentation I gave at QCon London is now available on the InfoQ site.  This is the same presentation as the one I gave at Skillsmatter in March, but the questions are different.  Plus since I'm winging it every time, I probably cover slightly different things or explain some stuff better / worse.

Interviewed for InfoQ at QCon London

I was flattered to be interviewed for InfoQ at QCon London.  It was a fun interview actually, and didn't feel anything like the half an hour it actually took.  In it, I get to talk about Agile at LMAX, the Disruptor (of course) and diversity in IT (again).

Why Open Source Your Secrets

Here's a video of my Open Conference session on the business benefits of open sourcing your software.  Given that the conference was at a weekend and had a very intimate feel, I think I was a teeny bit more honest than I usually am.  Enjoy.

Why Open Source Your Secrets

View more presentations from Trisha Gee.

QCon London 2012

I'm late with my write-up of QCon, and what's worse, it will be partial - "sadly" I was in Lanzarote on a training week with the running club from the Thursday (8th) so I missed most of it.  A sacrifice I had to make for 7 days in the sunshine….

Firstly, me me me

I presented the talk I previewed at Skillsmatter the previous week, something I was calling the User's Guide to the Disruptor, but actually turned out to be how-can-Trish-fill-95-slides-with-pictures-and-finish-in-under-40-minutes.

The audience was different to the Skillsmatter event, not surprisingly.  What was surprising is that I expected people at the conference to be less aware of the Disruptor, and those who came to the Disruptor-only LJC event to have had more exposure to it.  It was a (pleasant) surprise to see how many of the standing-room-only audience had not only heard of the Disruptor but had read stuff about it (I always love it when people have read my blog), played with it and were even using it in anger.

Because of that, I think if anything the talk did not go into enough detail, or enough new stuff, to please everyone.  Tough crowd!  But it was gratifying to hear the audience correct me in some of my answers, and answer other people's questions - it's always nice to know people are listening.

Of course, I will post a link to the presentation when it's available.  For now only the slides are online.

I enjoyed QCon

QCon was noticeably different to the other conferences I've been to in the last six months.  For one, it's not a Java conference - sure, I was hosted on the Java track, but QCon is wider-ranging than just one technology platform.  I'm not sure if it's because of this, or because it was based in the notoriously impatient London, but I felt like there was a message of "Look, let's just get stuff done, OK?".  Ultimately we get paid to deliver stuff for the business, and since my favourite question is always "but what are we trying to achieve?" I like to hear ideas around how to actually deliver.  Don't get me wrong, I've loved the technology conferences - I like to hear about new stuff I had no idea about, and I really like the community vibe from them.  But it's a nice change to be shaken up into thinking exactly why we do all this.

The Data Panorama

Firstly, Rebecca Parsons and Martin Fowler from my old employer ThoughtWorks put Big Data into perspective.  Previously I hadn't really cared about it - we process lots of data at LMAX but we don't really have to dig into it, so Big Data is not top of my "oooh I really worry about that" problems.  There were quite a few interesting points I took out of this:

 - In the past, it was easy (and possibly even correct) to model the whole application based on the data you were collecting or manipulating (and probably storing in a relational database).  These days it's not just the data from your app you need to worry about (and that can get big enough), but also all the news, blogs, twitter, and Facebook stuff generated by you and about you.  In addition, your data might not even be located with your app - the cloud has made the physical question of location redundant.  All of this pushes you towards an architecture which has to separate data from the application, and forces you to ponder your design.  I heard good arguments for Domain Driven Design here, which is nice because we like that at LMAX.

 - Reporting and analytics on Big Data must be more fluid.  There's so much data about you, your users, your application, out there that you don't even know the questions to ask.  Instead you want to be able to spot patterns in data you didn't even know you had,  I thought this was dead interesting - I studied Computer Science and Artificial Intelligence at university, and we were told data mining using AI was going to be fundamental to companies who wanted to be on the bleeding edge.  Only now is it looking like people realise it's becoming that important.

 - Martin referred to Data Scientists and said he was suspicious of scientists.  I'd been reading The Black Swan on the train in that morning and couldn't help but wonder if he's read the same thing - that was talking about how you should treat someone with suspicion if they suggest you can apply science and logic to anything that is… well, actually to anything other than actual science (by which he meant physics I believe).  By saying it's a science you suggest it's predictable and follows rules.  And if it was predictable, it wouldn't be hard.  Big Data is anything other than predictable - the data could be corrupt (it's safest to assume some of it is); it's generated by people (and we all know how fickle they are); and if you're collecting lots of it practically randomly, then cause/effect/correlation are not guaranteed.  So Martin suggests the term Data Journalist.  There was a storm on Twitter which suggested a certain amount of disagreement with this term, but I like it.  But then, I like to pretend I'm a writer and not a programmer.

 - They gave some examples of using data to drive economic growth (e.g. Kenya) - what I thought was interesting about this is that it was a win-win situation - expose the data to grow technology skills, but get a lot of interesting/useful/socially responsible applications in return.

 - Something I liked was about the idea of embracing "bad" data, stuff that's not trustworthy for whatever reason.  You can assume that the good data overwhelms the bad but you can't be sure.  By looking for more fluffy patterns, vague correlations, in your data, you might expose something interesting even if the data's not "correct".  As humans, we can't expect that we won't make a mistake - it's better assume we will and work out how to deal with it.

 - There was a call to not passively accept requirements, but to play an active role.  I like to think this ties in to my post about working with your customer.  But then I would.

 - I got a lot from this keynote, even if it was just a feeling of "I knew it!  I was right!". 

Highly available systems in Erlang

Joe Armstrong was a very interesting person to listen to, clearly someone who's been there and done that. Even without the presentation, the slides are interesting in their own right as they contain a lot of information and guidelines.

The points I took away are:

 - If part of the system fails, it's not up to that part to fix itself.  You need special help to deal with failures.  If you fell over with a heart attack, you wouldn't try and heal yourself, you'd get a medic

 - Isolation between threads/programs will mean that those different things cannot interfere with each other (i.e. no shared state).

 - My favourite quote was "If you make things synchronous you'll bugger things up".  In theory, it's so much easier for us humans (programmers) to think synchronously.  But whenever you design your system to by asynchronous, you find that your system actually becomes simpler and not more complicated.

JVM performance optimizations at Twitter's scale

 - I had a terrible view in the fully packed room, so I was just picking up phrases. I heard Attila mention the Uncanny Valley, a phenomenon I was introduced to by my sister when she was studying her Cybernetics PhD (Google it, I found it fascinating).  

 - There was a lot of really useful information about how the Java GC works.  It seemed to back up the (rough) premise we work on - stuff that's very short-lived is fine, and stuff that lasts "forever" is fine - it's the stuff in between which  cripples your system when it keeps getting shoved around.

Decisions Decisions

Dan North was, as always, an excellent speaker.  He entertained us but got us all thinking.  Five years ago at QCon London (my very first conference ever, and the thing that motivated me to start a professional blog), I saw Dan speaking and I was inspired to think about my working practices.  He was talking about BDD at the time, which was a relatively new concept to me.  I came away from that QCon with clearer ideas of what awesome development practices should look like.  Never did I imagine that five years later, not only would I be working in an environment which follows a lot of those practices (and pioneers many more) but that I would actually be speaking at the same conference.

I've come a long way since then, and of course, Dan isn't talking about the same things either.  I've been to a lot of conferences this year, and I heard nothing really preaching to the "meta-agile" LMAX (still have a post pending about our agile practices...).  When it comes to agile, there seem to be very few people who we can learn off - don't get me wrong, there are lots of things we want to improve on, which is why we're looking for people to learn off.  But most people are still preaching TDD and we want to know "what next?".

Well, Dan took everything we thought we knew, and ripped it to pieces.  Taught us to challenge everything we think we know.

It was irritating actually because he had no answers.  But he did tell us that the answers we think we already have might not be the right ones….

Well worth watching his talk when it comes online, I really can't summarise it here.

Developers Have a Mental Disorder

I nearly missed this ending keynote.  I'm so glad I didn't, Greg Young is awesome at ranting.  He said developers have a disease - we overcomplicate things, when we try to simplify things.  We want to abstract stuff, we love to look for patterns and reuse when actually sometimes we just need to solve the problem.

In my notes I have "People come to conferences for answers, when they should be remembering to use their brains".  I assume that's a quote from him, and not a comment I thought of at the time, but I wholeheartedly agree with it - if a shiny new technology solved your problems, you'd be out of a job.  Your job is to take the problem and figure out how to solve it.  Not to drag and drop an answer into place.

Another talk that I can't do justice to, watch the video when it comes online.

…and finally….

At the end of the day, the Atlassian-sponsored community night was awesome.  I got to chat to (be prepared for gratuitous name dropping here) Martin Fowler; my old friend Simon Brown; my ubiquitous LJC colleagues Martijn and John; the Zero Turnaround guys; a couple of ex-colleagues from Evolution / Detica; a heap of LMAX and ex-LMAX guys; and, of course, bundles and bundles of new interesting people.

Summary (i.e the short version for those who can't be bothered to read this whole post)

Maybe it's wishful thinking, but the messages I took from QCon were:

• If you want to solve the problem your business has, you might want to model your system around their world.  Funny, that sounds suspiciously like Domain Driven Design.
• Synchronous is bad, mmmkay? 
• Hardcore understanding of what the computer is really doing seems to be coming back into fashion.  Hmmm, I wonder who started that…? (tongue firmly in cheek, we can't have been the only ones)
• "You can't tune something you don't understand" - testing and monitoring is kinda important.
• Our business is all about trade offs.  There is no perfect solution, they pay us because it's very very hard to work out something that's Good Enough.
• Ultimately it feels like back to basics: understand the problem; model the domain, and have sympathy for the hardware that's running your solution.

My Corporate Bit

QCon overall turned out to be a bit of an LMAX fest in the end, with Mike & Martin and Andy Stewart (our Chief Lord Business Analyst), all giving presentations there as well as me.  It's nice to be on home turf, and it's very cool to see that we have such a range of things to talk about that so many of us are invited to speak.

PS

I just found out there's a QCon in New York.  My invitation seems to have got lost in the post.  Don't suppose anyone wants me to speak there…?

Java Magazine: Intro to the Disruptor Part One

This month's Java Magazine features an article by yours truly, which is yet another intro to the Disruptor.  It's basically a summary of the stuff I've written in this blog, updated for version 2.7 - so the names of the classes should be up to date and the responsibilities follow the simplified pattern we use now.  If you were looking for an more recent version of my introduction blog posts, this article gives a reasonable overview.

This is intended as part one of a series, as it's a basic and high-level view with no code examples.  In fact, it probably could be used to document the C# version as well as the Java version, although I haven't taken a look at that for a while.  Next, I would like to give some more code examples of how you use it - as always, any suggestions welcome.

New Disruptor Presentation Unveiled to the LJC!

A few weeks ago, I presented my new "User's Guide to the Disruptor" talk to the London Java Community.  Since it was very kindly hosted at Skillsmatter, there is a video of the presentation available, and the slides are below.

Concurrent Programming Using the Disruptor

View more presentations from Trisha Gee

The presentation is a little different to the ones we've done before.  Previously we've gone into how it works and why it's fast.  This time I wanted to step back a little from the internals and show how real developers might actually use it.  The example is somewhat contrived, but the idea is to give some hints on how to break your problem down into something that will work with a Disruptor at the heart of it.

I thought the event went really well.  It was a tiny bit completely intimidating, as there were no lightning talks and I was the only attraction.  Seeing over a hundred people turn up after work, before beer, to hear you talk is a humbling experience.  Fortunately, I think the audience was perfect for the presentation - they had heard of the Disruptor but hadn't seen anything very detailed about it, so my walkthrough of how you might use it seemed to go down well.  I certainly got a lot of very sensible questions (which hopefully I've remembered to repeat for the benefit of the recording), and people had some good ideas about how and when to use it.

I ran the same presentation to a different audience at QCon London a week later, I'll post a link to that if/when it becomes available.

Upcoming speaking events

In theory, I am busy writing material for my upcoming speaking events, rather than writing terribly illuminating posts on my blog (see what I did there?).  In actuality I am being lazy and have pretty much taken January off for a recharge.

In the spirit of doing something which ticks both the event-speaking and blogging boxes, this is a quick update on the conferences I'm confirmed for so far.  Put the following dates in your diary - these are my first international solo speaking events:

7th March - QCon London - Concurrent Programming Using The Disruptor (sadly I can't stay for the whole conference as it clashes with the only holiday I had booked for 2012).
23rd May - GOTO Copenhagen - Concurrent Programming Using The Disruptor & War Stories.
25-26th May - GOTO Amsterdam - Concurrent Programming Using The Disruptor.

The presentation will be more of a user's guide to the Disruptor than anything we've done before.  An hour isn't a lot of time to cover all the functionality everyone might want to see, so I'm still trying to work out the balance between giving an introduction/overview for those who haven't seen it before, and going into some of the cool features that have been added since I first started blogging about it.  If there's anything you would particularly like to see covered, let me know - I'll put the most frequently requested things in there.

Ideally I'd run a workshop session at some point, but that will require quite a lot more preparation, so I'll only do that if there is interest in it (if someone wants to fly me somewhere interesting to do that so much the better!!).

Maybe I'll see you at one of these events?

Video of our JAX London session

At JAX London Mike and I presented "Understanding the Disruptor - A Beginner's Guide to Hardcore Concurrency".  This is the session we initially previewed to the London Java Community a few weeks earlier.  The content is the same, but the feel of the presentation was quite different to us - the venue for the LJC event was more intimate, and it was easier to interact with the audience.  At JAX, we were up on stage, which was pretty cool actually, but meant that it felt more like a lecture and it was less easy to connect with the audience.

We received some really great feedback on this presentation, and it was brilliant to see a lot of the speakers from JAX there watching us.

London Java Community Open Conference

Saturday was, hopefully, my last conference of the year.  My lucky readers should start to see some posts which are not simply me gushing about another opportunity to hang out with awesome people and learn about interesting "stuff".

Who wants to propose a session?

In many ways the London Java Community Open Conference was my favourite one so far, and not just because it's near home and I helped to organise it.  One of the awesome things about both Java One and Devoxx was the opportunity to travel, to see new places and to meet people you might not meet in London.  The scale, and the opportunities to meet key players in the Java world, were the things I probably appreciated the most from both of those conferences.  And you can tell from my posts I really enjoyed them.

But the LJC conference was probably perfect as my last one for 2011:

"How do you spell 'lightning'?"

• Being on home turf with awesome people who have really helped drag me kicking and screaming into the conference scene really brought home to me what an amazing year this has been.  This time last year, I felt I barely had the credentials to be a behind-the-scenes organiser for the LJC, and I didn't attend last year's conference because I wasn't sure how much I would get out of it.  This year, I'm at the conference giving two sessions, having made my international début already.
• With 120 people you feel like you can talk to everyone at some point if you want to.  I don't think I managed that, but I probably chatted in one way or another with maybe half the attendees.  The great thing about this is you see the wide variety of things people are working on - the technologies, the business problems, the team and company sizes, the methodologies. It's eye-opening and quite exciting.
• It seemed that a small, open conference like this drives content based on relevance.  We had no vendor pitches, although John was contractually obliged to mention Atlassian at least once every half hour (but as they paid for the beer, this was only fair).  People vote with their feet, and attend the sessions that they will get the most out of.  I liked this format a lot.
• I presented alone for the first time, not hiding behind Mike or Martin.  I found this surprisingly liberating.  I love working with those guys, but without them providing a safety net I actually found my confidence increasing as I realised I was perfectly capable of answering things I would probably have let them field.
• My second session was more workshop-like, and I wanted the audience to guide what we covered.  I thought it went really well, I loved letting the audience guide it, and I enjoyed giving it.
• I was honoured (and terrified) to be asked to be part of a Meet the Experts panel with people who actually know what they're talking about.  I was very very pleased to get away with not being asked any questions about being a girl or the length of my skirt.  We had a great discussion around writing high performance code, about different technologies and their applicability, and about the future of Java and JVM-based languages.
• I loved the venue.  The rooms were a nice range of sizes, all with projectors of course, but also whiteboard space and flip charts.  It looked like a 1960s version of the future - all shiny white surfaces and curving lines.  But it felt like a space to create and innovate in.
• Mike's Hacking the Open JDK session in the morning was excellent.  Again, this was another example of a workshop format working really well.  He gave some good background to compilers in general, and a good walk through of some of the code that's there at the moment.  He was happy to share stuff he'd learnt the hard way, and it made me want to get more involved in that side of things.
• The food was awesome.

Clearly people haven't had enough coffee yet.

My sessions were User's Guides to the Disruptor - a beginner's and a more advanced one.  The beginner's was an updated summary of the Disruptor stuff already covered in this blog - the ring buffer, writing to and reading from it, and configuration.  The second session covered  more advanced ideas: you don't need a ring buffer any more; cache lines and false sharing; worker pools; aggregate event handlers. This session was a lot less prepared and more collaborative, and I loved giving it.  Maybe that's my teacher-genetics coming out.

The aim of both sessions was to give people an idea of how to actually use the Disruptor - developers are definitely interested in how and why it works, but we've been evangelising a lot and now there are very sensible questions being asked about how to get it to do various things.  I'd like to run some more of these sessions in future, and to add more material to the blog.

And finally, a proper meal before more beer.

Back to the conference.  Were there any downsides?  Personally I had none.  If I had to suggest an improvement for next year, it's that we would like to see more novice speakers presenting.  The LJC has always been about nurturing this talent, and a small conference like this one with a friendly audience, many of whom you may already know, is a great place to practice.  The lightning talks showed the great variety (and abilities) of our members, it would be nice to expand this to full sessions.  As I found, filling a 30 minute session is not anywhere near as hard as you think!

I'm really looking forward to next year's LJC conference, and I'm totally buzzing from the positive vibe from this year's.

The LJC Associates modelling their lovely new t-shirts.  Mine fits!

Mike and I debut our new Disruptor presentation

Last Tuesday Mike and I unveiled our brand shiny new presentation: Understanding the Disruptor, a Beginner's Guide to Hardcore Concurrency.  This was a preview of the talk we'll be doing at JAX London on the 2nd November.

A video of the session is available, as are the slides.  I promise not to say "so" anywhere near as many times when I repeat my performance at JAX (is there anything more painful than watching yourself on video?).

I thought the session went really really well.  We got some great questions at the end, we had an audience that was engaged, and I was dead pleased we didn't lose anyone with the assembly language.  We had some very valuable feedback afterwards too.

As well as our presentation, there were three great lightning talks:

Somay Nakhal on Java Thread States - Somay gave a nice overview of thread lifecycles with code and some great diagrams.  I liked how he made this more applicable to the real world than the sort of book examples you get.

Ged Byrne on the shiny new LJC Book Club - Ged reminded us how great it is to read an actual, paper book.  How committing to reading page by page forces you to learn in a different way to jumping around internet references that might not give you the context you need.  I thought this was a great presentation with humour, and I liked the way he challenged us to "expand our minds".  Although the actual book he was reviewing was Oracle Coherence 3.5, I've decided I need to read Beautiful Software, which Ged quoted at the end of the talk.

Peter Lawrey on Common Java Misconceptions - A session which plays well with what we're trying to preach when we talk about Tackling Folklore.  He covered a few topics that are assumed to be "truth".  For example, dealing with garbage collection is not a mandatory part of writing Java - you could write GC-friendly code for a start.  Also it's naive to assume the JDK is written in an efficient way, anyone who's actually dug around it for a while will realise that newer, more efficient methods of programming have not been applied to all areas of the (massive) existing code base.  I think it's great to have people out there talking about this stuff, it's too easy to make assumptions and take things for granted.  The most important thing he said: "If you're told something, don't just believe it - test it yourself first".

All of us (me, Mike and the lightning talk presenters) got such a great response it has encouraged us at the LJC to try and push for more real developers presenting their experiences.  We have a lot of great presentations from vendors, but what's more applicable to Java guys and girls across the board is other developers sharing the problems they're trying to solve and how they go about that process.

I'm very much looking forward to presenting this again at JAX.

JavaOne 2011: Roundup

Having been back in London for a few days I've had some time to digest the madness that was last week.

My lasting impression of JavaOne is almost entirely positive.  Granted, it was my first major conference, so maybe I'm just not jaded yet.  But let me tell you what I loved about it (yes, I did cover some of these in my last post):

• First and foremost, the people.  I don't remember meeting a single grumpy person. Everyone I spoke to was there to get the most out of the experience, regardless of how many times they'd been previously.  In my experience, techies are not conditioned to be socially comfortable, yet introductions were made and the conversations flowed easily.
• Of course it wasn't just the attendees who were friendly, the staff and organisers were approachable and helpful, and it was nice to have people hanging around to direct you.
• One of the (few) advantages of having the event over multiple hotels was the outdoor space between them.  It's unfortunate that it rained,  but I really liked being able to hang around outside.  I especially liked that they had provided power points for your laptop, so you didn't have to be cooped up indoors to update your blog.  I also thought that having to walk between the venues was good exercise, given I didn't make it to the gym last week. However I can see why people want to move the event back to one central location.

• I personally found the sessions less useful than the networking.  I mostly attended the high-performance / concurrency sessions, but I probably should have been to ones about things I didn't know (e.g. other JVM languages).  My favourite session was Martijn's Diabolical Developer session, but then I'm a fan of stand-up comedy.  
• Another favourite was the Java Posse live podcast.  I'm ashamed to admit that I've never listened to one before (my excuse: I don't listen to podcasts at all because I don't have a long commute to work, which would be the ideal time).  I thought it was great how everyone felt like a part of the Java Posse and not a passive audience.

• I really felt that Oracle was trying its best to invoke the spirit of community.  Maybe that's because when I wasn't in sessions or having random conversations, I was in Java User Group or Java Community Process events.  It seems pretty clear to me that the inclusion of SouJava and the London Java Community in the JCP EC has shaken things up a bit.  But then, I would say that - I'm biased.  But the vibe I was getting across the whole conference was that having the community involved in this key decision-making organisation is a step forward.  I would even go so far as to claim that it inspired a number of other players to seriously consider getting on board rather than complaining from the sidelines.

• On a related note, I really enjoyed the final keynote on community. To me, that was an indication that Oracle takes our participation very seriously.  I liked that it was split into a lot of different interviews and panels, all quite short, which showcased the variety of the Java platform community.  I came out of that feeling pretty warm and fluffy about the future of our technology, and proud to be an active part of it.

Stuff I was not so bothered about:

• Vendor keynotes.  Pretty dull to be honest.  I understand why they had them but after the first one I didn't turn up to any more.
• The other bad points I already mentioned in the last post.

Personal highlights:

• Picking up the Duke's Choice Award (video - note: only seems to work in Firefox for me) for the Disruptor. Yeah yeah, I'll shut up about it at some point, let me have my moment of glory.
• Co-presenting a session on the Disruptor with Martin.  He invited me to stand up with him when we realised I could go to the event, and I was terrified of the thought.  So I did it.
• Meeting and being interviewed by the JDuchess ladies.  The video doesn't seem to be available, so maybe I was rubbish, but it was a cool thing for me all the same.
• Talking about the Disruptor for the Java Spotlight Podcast and the Oracle news guys (again, I can't find the content for this yet).

I had a really brilliant time, and would love to repeat the experience again as soon as possible!

JavaOne: Initial Observations

So I've been at JavaOne for the better part of three days, it's time to record some of my observations so far:

• The wireless access is rubbish.
• <Gross generalisation> technical people are not natural public speakers.  Makes me feel better about the presentations I'm going to be giving (see A Beginner's Guide to Hardcore Concurrency).
• The sessions are less useful than getting out and chatting.  I've had a really excellent time, I've met: people from other Java User Groups; the Duchess girls; other Duke Award winners; the Azul guys; guys (well, girls) from O'Reilly books; JCP members and many random and awesome people.
• Everyone thinks that Large is an acceptable default t-shirt size (it's not).  Vendors - if you're really serious about appealing to The Other Gender you need to stock XS, if not actual skinny tees.
• If you're running a conference, you should probably have your projection screens above the height of the audience members' heads
• People at JavaOne are dead friendly.  I've ended up in a lot of conversations just by virtue of standing alone for longer than 30 seconds.  It is noticeably easier to talk to people here than at the conferences I've been attending in London.  Not sure if that's a location thing or a domain thing.
• Socialising in London is great practice for this sort of event.  I am capable of taking advantage of free drink and still maintaining a conversation and staying upright in 6 inch heels.
• I miss American breakfasts.  I've been gorging myself on pancakes, biscuits and gravy, and eggs benedict.  I'll be calling my personal trainer as soon as I return.
• Haven't seen anything to contradict my view that San Francisco is not the Brit's typical view of California - the weather is rubbish.  London has been hotter and sunnier this week.
• Sharing an apartment with your CTO is not as weird as you might think.  Especially if you relegate him to the closet (no, that's not a euphemism).
• It's difficult to remember to Tweet or blog when you're totally engrossed in conversations with people.

Here's a photo of me representing LMAX as I pick up the Duke Award we won for the Disruptor:

 (Thanks to Martijn for taking the photo).

I was grabbed for an interview which should be available (un-edited - erk!) on java.net at some point, I'll post it when it's available (if it's not rubbish).

Disruptor 2.0 - All Change Please

Martin recently announced version 2.0 of the Disruptor - basically there have been so many changes since we first open-sourced it that it's time to mark that officially.  His post goes over all the changes, the aim of this article is to attempt to translate my previous blog posts into new-world-speak, since it's going to take a long time to re-write each of them all over again. Now I see the disadvantage of hand-drawing everything.

In the old world

This is an example of a configuration of the Disruptor (specifically a diamond configuration).  If none of this means anything to you, feel free to go back and refresh yourself on all the (now outdated) Disruptor details.

The most obvious changes over the last few weeks have been:

1. Updated naming convention
2. Integrating the producer barrier into the ring buffer
3. Adding the Disruptor wizard into the main code base.

The New World Order

You'll see the fundamentals are pretty much the same.  It's simpler, because the ProducerBarrier is no longer an entity in its own right - its replacement is the PublishPort interface, which is implemented by the RingBuffer itself.

Similarly the name DependencyBarrier instead of ConsumerBarrier clarifies the job of this object; Publisher (instead of Producer) and EventProcessor instead of Consumer also more accurately represent what these things do.  There was always a little confusion over the name Consumer, since consumers never actually consumed anything from the ring buffer. It was simply a term that we hoped would make sense to those who were used to queue implementations.

Not shown on the diagram is the name change of the items in the RingBuffer - in the old world, we called this Entry, now they're an Event, hence EventProcessor at the other end.

The aim of this wholesale rename has not been to completely discredit all my old blogs so I can continue blogging about the Disruptor ad infinitum. This is far from what I want - I have other, more fluffy, things to write about.  The aim of the rename is to make it easier to understand how the Disruptor works and how to use it. Although we use the Disruptor for event processing, when we open sourced it we wanted it to look like a general purpose solution, so the naming convention tried to represent that.  But in fact the event processing model does seem more intuitive.

No more tedious wiring
Now the Disruptor wizard is part of the Disruptor itself, my whole post on wiring is pretty pointless - which is good, actually, because it was a little involved.

These days, if you want to create the diamond pattern (for example the FizzBuzz performance test), it's a lot simpler:

DisruptorWizard dw = new DisruptorWizard<FizzBuzzEvent>(
                         ENTRY_FACTORY, 
                         RING_BUFFER_SIZE, 
                         EXECUTOR,
                         ClaimStrategy.Option.SINGLE_THREADED,
                         WaitStrategy.Option.YIELDING);
FizzBuzzEventHandler fizzHandler = 
                         new FizzBuzzEventHandler(FIZZ);
FizzBuzzEventHandler buzzHandler = 
                         new FizzBuzzEventHandler(BUZZ);
FizzBuzzEventHandler fizzBuzzHandler = 
                         new FizzBuzzEventHandler(FIZZ_BUZZ);

dw.handleEventsWith(fizzHandler, buzzHandler)
  .then(fizzBuzzHandler);

RingBuffer ringBuffer = dw.start();

Note there is a Wiki page on the Disruptor Wizard.

Other changes: performance improvements
As Martin mentions in his post, he's managed to significantly improve the performance (even more!) of the Disruptor in 2.0.

The short version of this is that there is a shiny new class, Sequence, which both takes care of the cache line padding, and removes the need for memory barriers.  The cache line padding is now done slightly differently because, bless Java 7's little cotton socks, it managed to "optimise" our old technique away.

I'll leave you to read the details over there, in this post I just wanted to give a quick summary of the changes and explain why my old diagrams may no longer be correct.

Dissecting the Disruptor: Demystifying Memory Barriers

My recent slow-down in posting is because I've been trying to write a post explaining memory barriers and their applicability in the Disruptor. The problem is, no matter how much I read and no matter how many times I ask the ever-patient Martin and Mike questions trying to clarify some point, I just don't intuitively grasp the subject. I guess I don't have the deep background knowledge required to fully understand.

So, rather than make an idiot of myself trying to explain something I don't really get, I'm going to try and cover, at an abstract / massive-simplification level, what I do understand in the area.  Martin has written a post going into memory barriers in some detail, so hopefully I can get away with skimming the subject.

Disclaimer: any errors in the explanation are completely my own, and no reflection on the implementation of the Disruptor or on the LMAX guys who actually do know about this stuff.

What's the point?
My main aim in this series of blog posts is to explain how the Disruptor works and, to a slightly lesser extent, why. In theory I should be able to provide a bridge between the code and the technical paper by talking about it from the point of view of a developer who might want to use it.

The paper mentioned memory barriers, and I wanted to understand what they were, and how they apply.

What's a Memory Barrier?
It's a CPU instruction.  Yes, once again, we're thinking about CPU-level stuff in order to get the performance we need (Martin's famous Mechanical Sympathy).  Basically it's an instruction to a) ensure the order in which certain operations are executed and b) influence visibility of some data (which might be the result of executing some instruction).

Compilers and CPUs can re-order instructions, provided the end result is the same, to try and optimise performance.  Inserting a memory barrier tells the CPU and the compiler that what happened before that command needs to stay before that command, and what happens after needs to stay after.  All similarities to a trip to Vegas are entirely in your own mind.

The other thing a memory barrier does is force an update of the various CPU caches - for example, a write barrier will flush all the data that was written before the barrier out to cache, therefore any other thread that tries to read that data will get the most up-to-date version regardless of which core or which socket it might be executing by.

What's this got to do with Java?
Now I know what you're thinking - this isn't assembler.  It's Java.

The magic incantation here is the word volatile (something I felt was never clearly explained in the Java certification).  If your field is volatile, the Java Memory Model inserts a write barrier instruction after you write to it, and a read barrier instruction before you read from it.

This means if you write to a volatile field, you know that:

1. Any thread accessing that field after the point at which you wrote to it will get the updated value 
2. Anything you did before you wrote that field is guaranteed to have happened and any updated data values will also be visible, because the memory barrier flushed all earlier writes to the cache.

Example please!

So glad you asked.  It's about time I started drawing doughnuts again.

The RingBuffer cursor is one of these magic volatile thingies, and it's one of the reasons we can get away with implementing the Disruptor without locking.

The Producer will obtain the next Entry (or batch of them) and do whatever it needs to do to the entries, updating them with whatever values it wants to place in there.  As you know, at the end of all the changes the producer calls the commit method on the ring buffer, which updates the sequence number.  This write of the volatile field (cursor) creates a memory barrier which ultimately brings all the caches up to date (or at least invalidates them accordingly).  

At this point, the consumers can get the updated sequence number (8), and because the memory barrier also guarantees the ordering of the instructions that happened before then, the consumers can be confident that all changes the producer did to to the Entry at position 7 are also available.

...and on the Consumer side?

The sequence number on the Consumer is volatile, and read by a number of external objects - other downstream consumers might be tracking this consumer, and the ProducerBarrier/RingBuffer (depending on whether you're looking at older or newer code) tracks it to make sure the the ring doesn't wrap.

So, if your downstream consumer (C2) sees that an earlier consumer (C1) reaches number 12, when C2 reads entries up to 12 from the ring buffer it will get all updates C1 made to the entries before it updated its sequence number.

Basically everything that happens after C2 gets the updated sequence number (shown in blue above) must occur after everything C1 did to the ring buffer before updating its sequence number (shown in black).

Impact on performance
Memory barriers, being another CPU-level instruction, don't have the same cost as locks  - the kernel isn't interfering and arbitrating between multiple threads.  But nothing comes for free.  Memory barriers do have a cost - the compiler/CPU cannot re-order instructions, which could potentially lead to not using the CPU as efficiently as possible, and refreshing the caches obviously has a performance impact.  So don't think that using volatile instead of locking will get you away scot free.

You'll notice that the Disruptor implementation tries to read from and write to the sequence number as infrequently as possible.  Every read or write of a volatile field is a relatively costly operation. However, recognising this also plays in quite nicely with batching behaviour - if you know you shouldn't read from or write to the sequences too frequently, it makes sense to grab a whole batch of Entries and process them before updating the sequence number, both on the Producer and Consumer side. Here's an example from BatchConsumer:

long nextSequence = sequence + 1;

while (running)

{

try

{

final long availableSequence = consumerBarrier.waitFor(nextSequence);

while (nextSequence <= availableSequence)

{

entry = consumerBarrier.getEntry(nextSequence);

handler.onAvailable(entry);

nextSequence++;

}

handler.onEndOfBatch();

sequence = entry.getSequence();

}

...

catch (final Exception ex)

{

exceptionHandler.handle(ex, entry);

sequence = entry.getSequence();

nextSequence = entry.getSequence() + 1;

}

}

(You'll note this is the "old" code and naming conventions, because this is inline with my previous blog posts, I thought it was slightly less confusing than switching straight to the new conventions).

In the code above, we use a local variable to increment during our loop over the entries the consumer is processing.  This means we read from and write to the volatile sequence field (shown in bold) as infrequently as we can get away with.

In Summary
Memory barriers are CPU instructions that allow you to make certain assumptions about when data will be visible to other processes.  In Java, you implement them with the volatile keyword.  Using volatile means you don't necessarily have to add locks willy nilly, and will give you performance improvements over using them.  However you need to think a little more carefully about your design, in particular how frequently you use volatile fields, and how frequently you read and write them.


PS Given that the New World Order in the Disruptor uses totally different naming conventions now to everything I've blogged about so far, I guess the next post is mapping the old world to the new one.

Dissecting the Disruptor: Why it's so fast (part two) - Magic cache line padding

We mention the phrase Mechanical Sympathy quite a lot, in fact it's even Martin's blog title.  It's about understanding how the underlying hardware operates and programming in a way that works with that, not against it.

We get a number of comments and questions about the mysterious cache line padding in the RingBuffer, and I referred to it in the last post.  Since this lends itself to pretty pictures, it's the next thing I thought I would tackle.

Comp Sci 101
One of the things I love about working at LMAX is all that stuff I learnt at university and in my A Level Computing actually means something.  So often as a developer you can get away with not understanding the CPU, data structures or Big O notation - I spent 10 years of my career forgetting all that.  But it turns out that if you do know about these things, and you apply that knowledge, you can come up with some very clever, very fast code.

So, a refresher for those of us who studied this at school, and an intro for those who didn't.  Beware - this post contains massive over-simplifications.

The CPU is the heart of your machine and the thing that ultimately has to do all the operations, executing your program.  Main memory (RAM) is where your data (including the lines of your program) lives.  We're going to ignore stuff like hard drives and networks here because the Disruptor is aimed at running as much as possible in memory.

The CPU has several layers of cache between it and main memory, because even accessing main memory is too slow.  If you're doing the same operation on a piece of data multiple times, it makes sense to load this into a place very close to the CPU when it's performing the operation (think a loop counter - you don't want to be going off to main memory to fetch this to increment it every time you loop around).

The closer the cache is to the CPU, the faster it is and the smaller it is.  L1 cache is small and very fast, and right next to the core that uses it.  L2 is bigger and slower, and still only used by a single core.  L3 is more common with modern multi-core machines, and is bigger again, slower again, and shared across cores on a single socket.  Finally you have main memory, which is shared across all cores and all sockets.

When the CPU is performing an operation, it's first going to look in L1 for the data it needs, then L2, then L3, and finally if it's not in any of the caches the data needs to be fetched all the way from main memory.  The further it has to go, the longer the operation will take.  So if you're doing something very frequently, you want to make sure that data is in L1 cache.

Martin and Mike's QCon presentation gives some indicative figures for the cost of cache misses:

Latency from CPU to...	Approx. number of CPU cycles	Approx. time in nanoseconds
Main memory		~60-80ns
QPI transit (between sockets, not drawn)		~20ns
L3 cache	~40-45 cycles,	~15ns
L2 cache	~10 cycles,	~3ns
L1 cache	~3-4 cycles,	~1ns
Register	1 cycle

If you're aiming for an end-to-end latency of something like 10 milliseconds, an 80 nanosecond trip to main memory to get some missing data is going to take a serious chunk of that.

Cache lines

Now the interesting thing to note is that it's not individual items that get stored in the cache - i.e. it's not a single variable, a single pointer.  The cache is made up of cache lines, typically 64 bytes, and it effectively references a location in main memory.  A Java long is 8 bytes, so in a single cache line you could have 8 long variables.

(I'm going to ignore the multiple cache-levels for simplicity)

This is brilliant if you're accessing an array of longs - when one value from the array gets loaded into the cache, you get up to 7 more for free.  So you can walk that array very quickly.  In fact, you can iterate over any data structure that is allocated to contiguous blocks in memory very quickly.  I made a passing reference to this in the very first post about the ring buffer, and it explains why we use an array for it.

So if items in your data structure aren't sat next to each other in memory (linked lists, I'm looking at you) you don't get the advantage of freebie cache loading.  You could be getting a cache miss for every item in that data structure.

However, there is a drawback to all this free loading.  Imagine your long isn't part of an array.  Imagine it's just a single variable.  Let's call it head, for no real reason.  Then imagine you have another variable in your class right next to it.  Let's arbitrarily call it tail.  Now, when you load head into your cache, you get tail for free.

Which sounds fine.  Until you realise that tail is being written to by your producer, and head is being written to by your consumer.  These two variables aren't actually closely associated, and in fact are going to be used by two different threads that might be running on two different cores.

Imagine your consumer updates the value of head.  The cache value is updated, the value in memory is updated, and any other cache lines that contain head are invalidated because other caches will not have the shiny new value.  And remember that we deal with the level of the whole line, we can't just mark head as being invalid.

Now if some process running on the other core just wants to read the value of tail, the whole cache line needs to be re-read from main memory.  So a thread which is nothing to do with your consumer is reading a value which is nothing to do with head, and it's slowed down by a cache miss.

Of course this is even worse if two separate threads are writing to the two different values. Both cores are going to be invalidating the cache line on the other core and having to re-read it every time the other thread has written to it. You've basically got write-contention between the two threads even though they're writing to two different variables.

This is called false sharing, because every time you access head you get tail too, and every time you access tail, you get head as well.  All this is happening under the covers, and no compiler warning is going to tell you that you just wrote code that's going to be very inefficient for concurrent access.

Our solution - magic cache line padding

You'll see that the Disruptor eliminates this problem, at least for architecture that has a cache size of 64 bytes or less, by adding padding to ensure the ring buffer's sequence number is never in a cache line with anything else.

public long p1, p2, p3, p4, p5, p6, p7; // cache line padding

private volatile long cursor = INITIAL_CURSOR_VALUE;

public long p8, p9, p10, p11, p12, p13, p14; // cache line padding

So there's no false sharing, no unintended contention with any other variables, no needless cache misses.

It's worth doing this on your Entry classes too - if you have different consumers writing to different fields, you're going to need to make sure there's no false sharing between each of the fields.


EDIT: Martin wrote a more technically correct and detailed post about false sharing, and posted performance results too.