Links for 2015-02-26

  • JClarity’s Illuminate

    Performance-diagnosis-as-a-service. Cool.

    Users download and install an Illuminate Daemon using a simple installer which starts up a small stand alone Java process. The Daemon sits quietly unless it is asked to start gathering SLA data and/or to trigger a diagnosis. Users can set SLA’s via the dashboard and can opt to collect latency measurements of their transactions manually (using our library) or by asking Illuminate to automatically instrument their code (Servlet and JDBC based transactions are currently supported). SLA latency data for transactions is collected on a short cycle. When the moving average of latency measurements goes above the SLA value (e.g. 150ms), a diagnosis is triggered. The diagnosis is very quick, gathering key data from O/S, JVM(s), virtualisation and other areas of the system. The data is then run through the machine learned algorithm which will quickly narrow down the possible causes and gather a little extra data if needed. Once Illuminate has determined the root cause of the performance problem, the diagnosis report is sent back to the dashboard and an alert is sent to the user. That alert contains a link to the result of the diagnosis which the user can share with colleagues. Illuminate has all sorts of backoff strategies to ensure that users don’t get too many alerts of the same type in rapid succession!

    (tags: illuminate jclarity java jvm scala latency gc tuning performance)


    Binary message marshalling, client/server stubs generated by an IDL compiler, bidirectional binary protocol. CORBA is back from the dead! Intro blog post: Relevant: Steve Vinoski’s commentary on protobuf-rpc back in 2008:

    (tags: http rpc http2 netty grpc google corba idl messaging)

This entry was posted in Uncategorized. Bookmark the permalink. Both comments and trackbacks are currently closed.


  1. Keith Brady
    Posted February 27, 2015 at 09:55 | Permalink

    Vinoski’s commentary is good but I think he’s assuming a very naïve use of rpc mechanisms where the client just calls a method and doesn’t realise that it’s executing remotely. In practice, and definitely inside the ‘plex, I think it’s more common to be aware of the service remoteness and work with that knowledge (retries, backoffs, failovers etc). Clearly it does work in some sense or nobody’s gmail would be working (not to mention search, maps etc).

  2. Posted February 27, 2015 at 11:39 | Permalink

    hi Keith — I hope that knowledge and those rules-of-thumb can be encoded in gRPC somehow — they are vital. The lessons from CORBA are that making it too easy to “paper over” the remoteness of an RPC, so it appears like a local call, is what causes fragility in the long run.

  3. Keith Brady
    Posted February 28, 2015 at 11:58 | Permalink

    I haven’t tried gRPC yet but it’s the latest version of our internal RPC mechanism which I have used a fair bit. In the case of that system, there’s quite a bit of friction that prevents you treating it as just a kind of local call even after the connection is up and running. I think that while that makes the client’s life harder, it avoids the pitfall of taking it for granted that you describe. OTOH, we have a lot of internal training and example code that produces good practices, I’m not certain they would emerge automatically without that.