Connecting Python to a Java Rest Servlet using POST

Leeland's picture

Pulled my hair out for too long to admit to over this. I had to tie a Python script to a servlet but the amount of data was too large for a GET. I wrote the whole thing in a quick sprint but then it didn't work. It worked if I pumped data to it via a GET request. But, when I shifted to a POST it stopped working. After a lot of RTM and Googling I still didn't see my error. After a coffee and a walk I looked at it and did a head slap. In the Python code I had set the Content-type to "text/plain;charset=UTF-8".

I did learn a number of things and so I thought it would be worth POSTing my notes... ;^)

So you can easily send data to servlets from Python using either GET or POST using the httplib and urllib libraries. Here is a real simple example of sending a POST request from Python:

>>> import httplib, urllib
>>> params = urllib.urlencode({'action': 'getbacon', 'account': '1234'})
>>> headers = {"Content-type": "application/x-www-form-urlencoded",
...            "Accept": "text/plain"}
>>> conn = httplib.HTTPSConnection("server.com:8443")
>>> conn.request("POST", "/kitchen/query", params, headers)
>>> response = conn.getresponse()
>>> print response.status, response.reason
200 OK
>>> data = response.read()
>>> conn.close()

This is by far the easiest way to shoot data at a servlet you control. The real gotcha here is that THIS ONLY WORKS because the "Content-type" is set to "application/x-www-form-urlencoded".

On the server end doing it this way all the parameters are available in a servlet using the same method HttpServletRequest.getParameter(String param) regardless of the request style (GET or POST). In fact it takes a bit of an effort to figure out if the parameter came from the POST body or the URL as they are all just available. Here is an example of reading values in:

public void doPost(HttpServletRequest request, HttpServletResponse response)
  throws ServletException, IOException {

  String action = request.getParameter("action");
  String account = request.getParameter("account");

  // Work with the data...
}

Again this only works if the POST data is encoded as key-value pairs of content type "application/x-www-form-urlencoded" like when fed by a standard HTML form.

To use a more complex encoding scheme like say a JSON data stream for the post data, a custom decoder is required to process the raw datastream using the style:

BufferedReader reader = request.getReader();

Here the POST data from Python needs to be formated using JSON style:

>>> import httplib, urllib
>>> body = r'{"jsondatastr":["data"],"action":["getbacon"]}'
>>> headers = {"Content-type": "application/json",
...            "Accept": "text/plain"}
>>> conn = httplib.HTTPSConnection("server.com:8443")
>>> conn.request("POST", "/kitchen/query", body, headers)
>>> response = conn.getresponse()
>>> print response.status, response.reason
200 OK
>>> data = response.read()
>>> conn.close()

Here is a JSON POST processing example (uses org.json package )

public void doPost(HttpServletRequest request, HttpServletResponse response)
  throws ServletException, IOException {

  StringBuffer jb = new StringBuffer();
  String line = null;
  try {
    BufferedReader reader = request.getReader();
    while ((line = reader.readLine()) != null)
      jb.append(line);
  } catch (Exception e) { /*report an error*/ }

  try {
    JSONObject jsonObject = new JSONObject(jb.toString());
  } catch (ParseException e) {
    // crash and burn
    throw new IOException("Error parsing JSON request string");
  }

  // Work with the data using methods like...
  // int someInt = jsonObject.getInt("intParamName");
  // String someString = jsonObject.getString("stringParamName");
  // JSONObject nestedObj = jsonObject.getJSONObject("nestedObjName");
  // JSONArray arr = jsonObject.getJSONArray("arrayParamName");
  // etc...
}

For more information on JSON Java libraries you might want to check out: A better Java JSON library?

GOTCHA: It is worth noting that you can only get the post data from request.getReader once.

Thread Slivers eBook at Amazon