XML-RPC For Objective Caml

Shawn Wagner shawnw@speakeasy.org

Part I
Manual

1  Introduction

1.1  XML-RPC

XML-RPC is a protocol for remotely invoking functions in different programs (Usually on different computers) than where the calling code is. It's a pretty simple protocol compared to others like CORBA, DCOM, or even Java RMI. It's also now available for use from Ocaml. Currently, you can only call remote procedures, you can't serve requests for procedures from other programs. That's coming soon, though. More information about it can be found at http://www.xmlrpc.com.

1.2  Requirements

I use Ocaml 3.0.4. Older versions might not work. Version 2 certainly won't work.

This XML-RPC library depends on many others. It uses some directly, and others are used by libraries that it does use. They're all needed. They can all be found at the Ocaml Link Database (http://www.npc.de/ocaml/). There are a lot of them, true, but that's the price you pay for me not wanting to reimplement the wheel.
findlib
A lovely package-management system. I use it, and all of the other libraries below use it too.

pxp
The Polymorphic XML Parser. I use it to parse and validate the xml sent by the remote server in response to a request.

netclient
I use this library's Http_client module to do the talking to remote servers.

netstring
Used for the Base64 encoding.

xstr
Used for some odds and ends, and by other libraries.

equeue
Used by Http_client.

pcre
Perl Compliant Regular Expressions. Used by oxridl and the other libraries.

stew
My general utility library. Used by oxridl.

1.3  Compiling and Installing

This is easy. Run make, run make opt if you want the native-code version, and finally run make install. Most of the hard work in finding out what directory to install stuff in is done by findlib. The oxridl program is installed in /usr/local/bin by default. You'll have to change the Makefile to change that. I plan on adding autoconf support sometime to make this easier.

1.4  License

This library is released under the terms of the Gnu Lesser General Public License, and the oxridl compiler is released under the terms of the Gnu General Public License. I believe that all the libraries it uses are released under licenses that are compatible with the GPL; if not, please let me know.

2  Calling Remote Procedures

There are two ways to invoke remote procedures from Ocaml. The first, which is high-level and usually saves you from having to deal with any of the XML-RPC stuff directly, is obviously preferred over the second, low-level interface described in part II below unless you have no other choice.

With the high-level interface, you create an IDL file describing the Ocaml type signature of a method, the name that you want to call it with, and its remote host URL and method name (Which can be different from what you call it in Ocaml). Then you run oxridl on it, to produce a pair of .mli and .ml files that export the functions in the IDL file and implement them, doing all the low-level dirty work behind the scenes.

2.1  The IDL

Each IDL file has one or more remote method descriptions. The descriptions are based on Ocaml type signatures. It's probably easiest to show an example:

remote time: unit → datetime = ``http://time.xmlrpc.org/RPC2'' ``currentTime.getCurrentTime''


This declares that the Ocaml function Foo.time (Where Foo is based on the name of the IDL file), which takes a unit argument and returns a dateTime.iso8601 (As an Ocaml string). The actual method lives on a server accessed by the URL http://time.xmlrpc.org/RPC2, and is called currentTime.getCurrentTime.

When you put that signature in a file foo.idl, and run orxidl on it, you can compile foo.ml to get Foo.time (). When you call that function, the remote server is queried, and what it returns is returned by the ocaml function. Spiffy, no?

2.1.1  Types

XML-RPC has a handful of basic data types, most of which correspond neatly to ocaml types. Table 1 covers the types used in the IDL, and what ocaml types they are mapped to.
XML-RPC type IDL type Ocaml type Notes
  unit unit Not in XML-RPC. See explanation below.
boolean bool bool  
string string string  
double float float  
dateTime.iso8601 datetime string I need to write functions to handle this kind of timestamp.
base64 base64 string This will encode and decode the data for you.
base64 rawbase64 string Does not encode or decode the data.
int or i4 int int XML-RPC requires 32-bit integers, which ocaml ints aren't!
int or i4 int32 int32 When you need full-sized XML-RPC ints.
struct struct XmlRPCTypes.xrs You have to use low-level accessors for structs.. See section 5.3
  raw XmlRPCTypes.t For using a XML-RPC type directly
array X array X array X is any of the above types.
array raw array XmlRPCTypes.t You have to use the low-level stuff for arrays of aggregate type.

The unit IDL type is used to indicate that a remote function either takes no arguments or returns no value.
Table 1: Mapping of XML-RPC, IDL, and Ocaml types

2.1.2  oxridl

The program that translates these IDL files to Ocaml code is called oxridl. It takes one or more IDL files as its command-line arguments. For each file foo.idl, it will create foo.mli and foo.ml, which respectively contain the signatures and stub code for calling the remote procedures. They need to be compiled and linked with programs that use those procedures.

If given a -o bar argument, bar.mli and bar.ml are used for every IDL file's generated code, instead of having a pair of files per IDL file.

It also understands --help and --version.

2.2  The low-level API

While this is described in part II, it's probably worth showing how to use the same XML-RPC service used in the IDL example with just the low-level API calls. So, here goes.
  1. First you have to create a XmlRPCClient.remote object. It takes two constructors, the URL of the server and the name of the method. So, let handle = new XmlRPCClient.remote ``http://time.xmlrpc.com/RPC2'' ``currentTime.getCurrentTime''

  2. Now that you have the object, you need to invoke the remote procedure via the object's call method. Arguments are passed to it in a list of XmlRPCTypes.t elements. This function takes no arguments, so we invoke it with an empty list. let result = handle#call []

  3. Finally, we have to get at the value of the returned XmlRPCTypes.t, with XmlRPCTypes.ml_of_datetime retval. Now, see how much easier the IDL interface is to use?

3  Serving Procedure Remotely

You can't do this yet. I'm working on it, though. Some ideas I'm mulling around:

3.1  Server design

There will be two layers to XML-RPC servers. The first, dispatch level, is where you register new functions. When the second, source level gets an XML-RPC request, it parses the XML of the request and passes that to the dispatch level, which figures out what happens, calls a handler, and passes the response back to the source, which sends it back to the remote caller. Pretty simple.

3.2  Sources

  1. Provide a way to to handle a incoming request for a procedure call in ocaml CGI programs, with the user's web server configured to run the CGI program when the proper URL is posted to. This is the easiest approach, and the most portable. It will happen. But it's pretty bad efficiency wise.

  2. Write an apache module that loads ocaml modules and calls routines in them to handle the HTTP requests that the module captures. I have a bad feeling that if I do this, it'll turn into a mod_ocaml like mod_perl.

  3. Write a small specialized web server just to handle the XML-RPC requests and have it call the relevant handlers for procedure requests directly. I have a bad feeling this'll turn into a full-blow web server.

  4. Find a small web server that someone's written in ocaml and hack the guts out of it to use the same approach as the previous item. I just saw one called wserver on the Caml Hump. Need to look into it and see if it's promising.
Out of these, the CGI will happen. One or more or the others might happen. The IDL will also be extended to make writing handlers for remote requests easier and minimize the low-level fiddling with XML-RPC types.

4  Future Plans

4.1  PXP

I'm probably not using PXP in the most effective way. At the moment, it parses and validates a methodResponse document, and then I walk its tree structure to extract the returned value. I have a feeling I can do the same with its spec and class stuff, but I need to read more docs and play around with it some more to do that. I'd never looked at PXP before starting this project, and just went with the SAX approach that I'm familiar with.

Ditching PXP is also a possibility. It's good, but it's big, and really bloats the size of programs using it. I might write a simple parser just for the XML-RPC document types (Or, more generally, something that reads a DTD and creates a parser just for it).

4.2  oxridl

Besides making it generate stubs for serving XML-RPC requests, I'd like to add a way to define the layout of structure types and automatically convert the struct to that ocaml struct type, rather than returning a XmlRPCTypes.xrs and requiring you to call the lookup functions.

With that will come vastly improving the IDL file parsing. Right now, it looks at each line in turn, matching it against a regular expression. It's a quick and dirty hack, and it could be improved vastly.

On the other hand, I had another idea. Instead of using a seperate IDL file and compiler for it, it might be possible to use caml4p to be able to define the remote procedures directly in ocaml source files by defining an syntax extender. Howerver, I have no idea how to do this yet, so it'll be on hold until I learn more about the pre-processor.

Part II
API Reference

Interface for module XmlRPCTypes

5  Interfacing XML-RPC and ocaml types


5.1  XML-RPC structs


An XML-RPC struct consists of zero or more key/value pairs, where the key is a string and the value is a t. You don't need to know how the structs are implemented internally (But if you're curious, as (string, t) Hashtbl.t at the moment), just use the functions described in section 5.3 to get at and access pairs.


type xrs

5.2  XML-RPC types


This type includes all the possible XML-RPC types, and the ocaml type they corespond to. Functions listed below allow for easy extraction of the the ocaml type, or creation of the XML-RPC type.

Note that for Base64, the string argument should not be actually base64-encoded by you. Use RawBase64 to hold data that's already thus encoded.


type t =
   | Unit
   | Int of int32
   | String of string
   | Boolean of bool
   | Double of float
   | Base64 of string
   | RawBase64 of string
   | DateTime of string
   | Array of t array
   | Struct of xrs

5.3  Structs again


These functions can be used to create new structs, and manipulate existing ones -- like, say, one returned by a remote function.
Create a new struct.
val make_struct: unit → xrs

Insert a new key/data pair into the struct
val add_elem: xrs → string → t → unit

Return the data for a key. Raises Not_found if the element isn't present
val find_elem: xrs → string → t

5.4  Miscellanious routines


This exception is thrown by many of the conversion functions in this module when passed a different type than what they're expecting


exception Bad_type

escape_cdata string turns <, > and & into XML/HTML entities
val escape_cdata: string → string

5.5  Converting types


5.5.1  Ocaml to XML-RPC


These functions take an argument of an ocaml type and return it stored in the corresponding XML-RPC type.


val xr_of_int : int → t
val xr_of_int32 : int32 → t
val xr_of_string : string → t
val xr_of_bool : bool → t
val xr_of_float : float → t

xr_of_base64 will store data that's not already base-64 encoded
val xr_of_base64 : string → t

But xr_of_rawbase64 does expect its argument to already be encoded.
val xr_of_rawbase64 : string → t
val xr_of_array : t array → t
val xr_of_struct: xrs → t

val skip: t

5.5.2  XML-RPC to Ocaml


These functions convert Ocaml reps of XML-RPC types into native Ocaml types. If passed a XmlRPCType.t of the wrong type for what you want to convert it to, they will raise XmlRPCType.Bad_type in revenge.


val ml_of_int : t → int
val ml_of_int32 : t → int32
val ml_of_string : t → string
val ml_of_float : t → float
val ml_of_bool: t → bool

ml_of_base64 will return the decoded data
val ml_of_base64 : t → string

And ml_of_rawbase64 will return the data still base64-encoded
val ml_of_rawbase64 : t → string

val ml_of_struct: t → xrs

val ml_of_datetime: t → string

val ml_of_array : t → t array
val ml_of_intarray: t → int array
val ml_of_int32array: t → int32 array
val ml_of_floatarray: t → float array
val ml_of_datetimearray: t → string array
val ml_of_base64array: t → string array
val ml_of_rawbase64array: t → string array
val ml_of_stringarray: t → string array
val ml_of_boolarray: t → bool array
val ml_of_structarray: t → xrs array

5.5.3  XML conversion


These functions are used to convert to and from XML
print_type t returns its argument as an XML-RPC value string
val print_type : t → string

parse_value node takes a PXP value node and returns its XML-RPC value. The node must be a <value>.
val parse_value: α Pxp_document.node → t

Interface for module XmlRPCDtd

6  The XML-RPC dtd


This string is the XML-RPC dtd. It's based on the one written by one Richard Moore, with comments stripped out. The original in included in the ocaml-xml-rpc distribution if you're interested.


val dtd: string

This is the parsed form, with the root node set to methodResponse. Don't depend on this name staying the same. I might change it to be more indicative of the root node setting.
val parsed_dtd: Pxp_dtd.dtd

Interface for module XmlRPCNet

7  Speaking to XML-RPC servers

The actual connection and communication with the remote server is done using the netclient library's Http_client module. All its caveats about thread safety apply. In the future I might make a thread-aware version that uses a different pipeline for each thread, in which case handler will become a function call.


handler is the Http_client.pipeline instance that's used to run all XML-RPC calls. Use the relevant methods on this to fine-tune stuff.
val handler: Http_client.pipeline

Interface for module XmlRPCClient

8  Remote method invocation

new remote xmlrpc-server method returns an object that can be used to execute the foreign method. Its simple_call method is convienience for a function that only takes one argument, and zero_call for functions without arguments. call must be used for all others.

While there's nothing stopping people from using this class directly, using functions generated by oxridl is much nicer and simpler, because it handles the conversion btween XML-RPC types and ml types automatically, for the most part. Please do so if possible.


8.1  Exceptions


This exception (Or XmlRPCTypes.Bad_type) is raised if there's technical problem with the processing -- can't connect to the server, the server returned something besides valid XML-RPC xml, or whatever.


exception Request_failed of string

This exception is raised if the request completes successfully, but returned a <fault> instead of a <params> in the <methodResponse> node. In other words, error generated from the server because of the request, not in the transaction. It holds a XmlRPCTypes.xrs with the fault information.


exception Request_fault of XmlRPCTypes.xrs

8.2  Remote calls


This is the string sent as the user-agent argument in HTTP headers.
val client_version: string

This class encapsulates a remote procedure. See above for details.
class remote: string → string → object
   method zero_call: unit → XmlRPCTypes.t
   method simple_call: XmlRPCTypes.t → XmlRPCTypes.t 
   method call: XmlRPCTypes.t list → XmlRPCTypes.t
end


This document was translated from LATEX by HEVEA.