Erlang is designed and used for distributed systems. As such, it is quite good at talking to itself. And as I'll show in the following, it is — at least in some cases — reasonably good at talking to other languages as well.
When would you want to do this?
(You can skip this section if you know.)
Interoperability between languages is useful when there's value to be had in the software in both ends. You might prefer to keep some functionality in Erlang because it's already written in that language; because it has better libraries for the task; or simply because the language supports that task better. The same goes for the other language: there are C libraries around for almost anything (and plenty of C/C++ legacy code), and for GUIs you might prefer e.g Java.
Or you have the same functionality implemented in both languages, and wish to do comparison tests on the two implementations.
Or the architecture is inherently distributed — e.g. a client-server configuration — so that there is no advantage to be had in building the two parts using the same language anyway.
In the following demonstration, we have a Java side with some freshly written software with interesting bugs, and an Erlang side with an interesting tool for discovering bugs and a nice-ish language for specifying tests.
The demo
What I intend to do:
- Define an interface;
- Write a Java function;
- Make it accessible from Erlang;
- Test it using Erlang.
Sounds like a lot of work? It needn't be. Not for you, anyway; most of the heavy lifting has already been done.
(If you don't care about exposition, and are just interested in the technical end result, here is the executable summary.)
An interface
First, we'll need to define an interface. For this, we use the IDL — Interface Description Language:
// File "foo.idl"
interface Foo {
string quuxinate(in string s);
};
We translate this into Java using the Erlang compiler:
erlc '+{be,java}' foo.idl
This results in a Java interface in Foo.java, as well as some stub code we'll be using.
A function in Java
For an implementation, let's say that quuxinate() is to reverse the string.
To add a twist, let's say that it breaks down under some rare, complex-ish circumstances, such as when there's a letter which occurs thrice in the input string — that'll be a bug we can find later, when we start testing it:
// File "FooImpl.java"
public class FooImpl extends _FooImplBase /* which implements Foo */ {
public String quuxinate(String s) {
int[] stats = new int[256];
for (int i=0; i<s.length(); i++) {
char c = s.charAt(i);
if (c<255 && ++stats[c] >= 3) throw new RuntimeException("WTF");
}
return new StringBuilder(s).reverse().toString();
}
}
Making the function accessible from Erlang
You may have noticed that we don't implement Foo directly, but rather extend an stub class which implements it.
This means that the class already knows how to talk to Erlang — specifically, it knows how to handle requests like a gen_server.
What is left is to establish connection to an Erlang node. There are two frameworks for this: CORBA and Erlang inter-node communication. CORBA is, I believe, the heavy-weight option, and I haven't worked with it; in the following, we'll use Erlang inter-node communication.
A runnning Erlang program (OS-level process) is often referred to as a 'node', because Erlang is designed for having multiple such programs be connected, in what is known as a 'cluster' of nodes.
It's not an exclusive club, either: nodes can be, and usually are, Erlang nodes, but other kinds can enter the mix — a C or Java node, for instance.
A bit of code is needed to run a Java program as a node; don't worry though, this is the only lengthy bit, and it can be generalized so that it doesn't have to refer to Foo:
// File "FooServer.java"
public class FooServer {
// The following is based on the program in lib/ic/examples/java-client-server/server.java
static java.lang.String snode = "javaserver";
static java.lang.String cookie = "xyz";
public static void main(String[] args) throws java.io.IOException, com.ericsson.otp.erlang.OtpAuthException {
com.ericsson.otp.erlang.OtpServer self = new com.ericsson.otp.erlang.OtpServer(snode, cookie);
System.err.print("Registering with EPMD...");
boolean res = self.publishPort();
if (!res) throw new RuntimeException("Node name was already taken.");
System.err.println("done");
do {
try {
com.ericsson.otp.erlang.OtpConnection connection = self.accept();
System.err.println("Incoming connection.");
try {
handleConnection(connection);
} catch (Exception e) {
System.err.println("Server terminated: "+e);
} finally {
connection.close();
System.err.println("Connection terminated.");
}
} catch (Exception e) {
System.err.println("Error accepting connection: "+e);
}
} while (true);
}
static void handleConnection(com.ericsson.otp.erlang.OtpConnection connection) throws Exception {
while (connection.isConnected() == true) {
FooImpl srv = new FooImpl();
com.ericsson.otp.erlang.OtpInputStream request= connection.receiveBuf();
try {
com.ericsson.otp.erlang.OtpOutputStream reply = srv.invoke(request);
if (reply != null) {
connection.sendBuf(srv.__getCallerPid(), reply);
}
} catch (Exception e) {
System.err.println("Server exception: "+e);
e.printStackTrace(System.err);
handleException(e, connection, null);
}
}
}
static void handleException(Exception e, com.ericsson.otp.erlang.OtpConnection connection, com.ericsson.otp.ic.Environment env) throws Exception {
// We'll improve on this later...
throw e;
}
}
Time to build and try out:
ERL_ROOT=`erl -noshell -eval 'io:format("~s\n", [code:root_dir()]), init:stop().'` # Or simply where Erlang is.
IC_JAR=`ls -1 $ERL_ROOT/lib/ic-*/priv/ic.jar`
JI_JAR=`ls -1 $ERL_ROOT/lib/jinterface-*/priv/OtpErlang.jar`
CLASSPATH=".:$IC_JAR:$JI_JAR"
javac -classpath "$CLASSPATH" *.java
epmd # Start Erlang port mapper daemon if it isn't already.
java -classpath "$CLASSPATH" FooServer
The Java node should now be running and registered as javaserver — ready to accept connections with the right cookie.
Let's test that:
erl -sname tester -setcookie xyz
> {ok,Host}=inet:gethostname().
> JavaServer = {dummy, list_to_atom("javaserver@"++Host)}.
> gen_server:call(JavaServer, {quuxinate, "Testing, 1-2-3"}).
The reply should be the reverse string:
"3-2-1 ,gnitseT"
And the bug we put there is working too:
> gen_server:call(JavaServer, {quuxinate, "Testing, 1 2 3"}).
** exception exit: {{nodedown,javaserver@flitwick},
{gen_server,call,
[{dummy,javaserver@flitwick},
{quuxinate,"Testing, 1 2 3"}]}}
in function gen_server:call/2
Property testing of Java code
Now then, what can we do with this setup?
One interesting thing that we can do is to apply one of the property-based testing tools to our Java function.
In the following, I'll be using Triq, but Quviq QuickCheck or PropEr could be substituted with only minor changes.
First, assuming that you haven't got Triq, but have git:
git clone git://github.com/krestenkrab/triq.git
(cd triq && ./rebar compile)
Then we are ready to write our test — namely, that given any (ASCII) string, quuxinate() should return the reverse string:
// File "test.erl"
-module(test).
-include_lib("triq/include/triq.hrl").
-export([main/0]).
prop_reverse(JavaServer) -> % The property
?FORALL(S, ascii_string(),
gen_server:call(JavaServer, {quuxinate, S})
== lists:reverse(S)).
ascii_string() -> % A data generator
list(choose(0,127)).
main() ->
{ok,Host}=inet:gethostname(),
JavaServer = {dummy, list_to_atom("javaserver@"++Host)},
triq:check(prop_reverse(JavaServer), 100), % Do the magic
init:stop(). % Shut down cleanly
Compile and run the test:
erlc -I triq/include -pa triq/ebin test.erl
erl -noshell -sname tester -setcookie xyz -pa triq/ebin -run test main
Triq will now generate a hundred random test cases, and verify the property for each case. After a dozen such tests, the bug is triggered:
...........Failed with: {exit,
{{nodedown,javaserver@flitwick},
{gen_server,call,
[{dummy,javaserver@flitwick},
{quuxinate,
[79,84,75,110,3,42,73,14,1,53,76,42,126,40,118,122,
74,2,58,34,42,98]}]}},
[{gen_server,call,2},
{test,'-prop_reverse/1-fun-0-',2},
{triq,check_input,4},
{triq,check_forall,6},
{triq,check,3},
{test,main,0},
{init,start_it,1},
{init,start_em,1}]}
Failed after 12 tests with {'EXIT',
{{nodedown,javaserver@flitwick},
{gen_server,call,
[{dummy,javaserver@flitwick},
{quuxinate,
[79,84,75,110,3,42,73,14,1,53,76,42,126,40,
118,122,74,2,58,34,42,98]}]}}}
The 22-character string has three '*'s (ascii value 42) in it, which was what triggered the bug.
Triq then proceeds to simplify the test case:
Simplified:
S = [33,33,33]
concluding that the string "!!!" is a locally-minimal failing test case.
Quite useful, isn't it? — and the amount of non-reusable code has been quite manageable (3 lines of IDL, 14 lines of implementation, 14 lines of test code, 1 line of Foo-specific code in FooServer).
(Speaking of which: you're free to use these snippets as you see fit; provided as-is and with no guarantees of anything, of course.)
I should mention at this point that property testing tools exist within the Java world as well — there exists at least one for Scala. I didn't find it particularly satifying to use, though, although I may have been unlucky; in any case, this is an alternative.
Erlang, IDL and exceptions
The Guide (that is, the Erlang IC (IDL compiler) User Guide) has this to say about handling of Java exceptions:
While exception mapping is not implemented, the stubs will generate some Java exceptions in case of operation failure. No exceptions are propagated through the communication.
Which means that, out of the box, these are our options for handling Java-side exceptions:
- Convert exceptions into values within
quuxinate(). - Return no result on exceptions — in which case the Erlang-side call will time out (after, as a default, 5 seconds).
- Close the connection — in which case the Erlang-side call will receive an error result immediately.
None of these options are especially satisfying.
So let's look at how to do this:
- Report exceptions to the caller as an
{error, Reason}reply.
It's not too difficult. What we need to do (according to the gen_server call protocol) is send a {RequestRef, Reply} tuple to the caller, where RequestRef is a reference which was included in the request and must be included in the response as well.
The main difficulty is one of access: at the point of error handling, we will need access to (1) the caller's PID and (2) the request reference.
I'd like to keep FooImpl and the connection-managing FooServer separate, so we need to add an accessor:
// File "FooImpl.java"
public class FooImpl extends _FooImplBase /* which implements Foo */ {
...
/** The request environment is exposed for error handling reasons. */
public com.ericsson.otp.ic.Environment getEnv() {
return _env;
}
}
That's it. We could instead have added one accessor for the caller PID and one for the request reference, but let's keep the complexity in the server class.
In the server class, instead of throwing an exception which causes the connection to be terminated, we will build and send an error reply:
// File "FooServer.java"
public class FooServer {
...
// in handleConnection(), replace
// handleException(e, connection, null);
// with:
handleException(e, connection, srv.getEnv());
...
// and replace handleException() with:
static void handleException(Exception e, com.ericsson.otp.erlang.OtpConnection connection, com.ericsson.otp.ic.Environment env) throws Exception {
// Write exception reply:
com.ericsson.otp.erlang.OtpOutputStream err_reply = new com.ericsson.otp.erlang.OtpOutputStream();
err_reply.write_tuple_head(2);
err_reply.write_any(env.getSref());
err_reply.write_tuple_head(2); // Construct return value {error, ErrorText}
err_reply.write_atom("error");
err_reply.write_string(e.toString());
connection.sendBuf(env.getScaller(), err_reply);
}
}
There; that's all.
If we test it again, we get a nicer behaviour:
> gen_server:call(JavaServer, {quuxinate, "Testing, 1 2 3"}).
{error,"java.lang.RuntimeException: WTF"}
Disclaimer
I ought to mention that I have only just discovered this interoperability option (two days ago, in fact, when I stumbled upon this article); it is only a few months ago that I wrote a custom socket server test program in Java, and corresponding driver code in Erlang, just to achieve what could be had far more easily using Erlang's IDL compiler.
It may therefore not be so easy to use in practice as the above make it seem.
(For one thing, the only type that I've used is string.)
Even so, I hope to have inspired others to try this out.
One way to do so is to have a look at the executable summary referred to earlier; it is the demo as a shell script.
Happy inter-language hacking!
Further technical notes (and speculation)
The IDL compiler supports generating code for Java, C and Erlang. Here is its documentation.
If you're writing Erlang port programs (like an Erlang driver, but running in a separate process), but have an interface which is evolving quicḱly enough, or you yourself are lazy enough, that writing and maiintaining the serialization and deserialization code has quite lost its attraction, then it might be tempting to look into using IDL for that interface, and have the boring bits code-generated for you.
The IDL compiler and the code it generates does not seem to have been designed for this, but as far as I can tell it appears to be achieveable.
