24 February 2012

Set up a VNC Server on Linux

I would like to reblog this: How To Setup Linux VNC Server In 3 Minutes. It is a good guide on  how to set up a VNC Server in Linux (via command line only).

While I'm on the topic I'll make a ranking of how good different programs/technologies (I have tried) to remotely login and get a graphical interface:

  1. Microsoft RDP. This protocol is probably the best. It can handle high latency rather good. It will adapt to screen size and copy commands always work. The big disadvantage is the high price (about one Windows license per concurrent RDP session).
  2. Go Global. It is also very fast, but also comes at a price. One drawback is that is not so common.
  3. VNC. This protocol is rather okay on handle latency. One disadvantage is that the screen size is fixed during startup.
  4. X11. The worst protocol to handle latency. This can totally destroy a user experience! The big advantage is that it is free and always works.
And there is bunch more I never tried...

09 February 2012

How to extract data from a Java heap dump

I will show how to extract data from the memory of a running Java application. Lets assume that this is the class with interesting data:
package se.lesc.blog.heap_extract;

import java.util.*;

/** Class that contain data to extract */
public class ArrayDataContainer {
    
    List<byte[]> arrays = new ArrayList<byte[]>();
    
    public String toString() {
        String result = "";
        for (int i = 0; i < arrays.size(); i++) {
            result += "Array " + i + ": " + new String(arrays.get(i)) + "n";
        }
        return result;
    }
}

An example main class that starts a Java process and populates the class with some data:
    public static void main(String[] args) throws Exception {
        ArrayDataContainer arrayDataContainer = new ArrayDataContainer();
        
        for (int i = 0; i < 20; i++) {
            String dataString = "This is my data " + i;
            arrayDataContainer.arrays.add(dataString.getBytes());
        }
        
        System.out.println(arrayDataContainer.toString());
        Thread.sleep(1000*300);
    }


Use your favorite tool to take a heap dump (for example jmap).

Open Java Visual VM (typically found in C:\Program Files\Java\jdk1.6.0_24\bin\jvisualvm.exe). It has an Object Query Language (OQL) feature that is very useful in this case. Go to File -> Load... Select Heap Dumps in the file open dialog and open the heap dump. Click on the "OQL Console" button.

Entry this query and press the execute button:
map(heap.objects("se.lesc.blog.heap_extract.ArrayDataContainer"), 
  function (it, index, array, result) {
    var res = '';
    for each (var element in it.arrays.elementData) {
      res += '<p/>';
      for each (var i in element) {
        res += i + ', ';
      }
    }
    return res;
  })

Copy the data from the Query Results and save it to a text file. Use this program to parse the file and recreate the List of byte[]:
package se.lesc.blog.heap_extract;

import java.io.*;
import java.util.*;

public class ArrayDataImporter {

    public static List<byte[]> importArrayList(InputStream in) throws Exception {
        List<byte[]> arrays = new ArrayList<byte[]>();
        
        BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(in));

        String line;
        while ((line = bufferedReader.readLine()) != null) {
            
            line = line.trim();
            if (line.isEmpty()) {
                continue;
            }

            List<Byte> array = new ArrayList<Byte>();
            
            String[] dataPoints = line.split(",");
            for (String dataPointString : dataPoints) {
                dataPointString = dataPointString.trim();
                
                int dataPoint = Integer.parseInt(dataPointString);
                array.add((byte) dataPoint);
            }
            
            byte[] byteArray = new byte[array.size()];
            for (int i = 0; i < array.size(); i++) {
                byteArray[i] = array.get(i);
            }
            
            arrays.add(byteArray);
        }
        
        return arrays;
    }
    

    public static void main(String[] args) throws Exception {
        ArrayDataContainer arrayDataContainer = new ArrayDataContainer();
        arrayDataContainer.arrays =
             ArrayDataImporter.importArrayList(
                 ArrayDataImporter.class.getResourceAsStream("query_results.txt"));
        System.out.println(arrayDataContainer);
    }
}

Things to note:
  1. The OQL help can be found at http://visualvm.java.net/oqlhelp.html.
  2. The OQL is mostly Javascript, so most of the "normal" Javascript functions will work.
  3. The heap.objects method targets every instance of a class (there was only 1 instance in this application).
  4. The it.arrays.elementData needs explaining. "it" is the iterator from the Javascript map method. "arrays" is the name of the attribute inside in my class ArrayDataContainer. "elementData" is the attribute inside ArrayList.
  5. The result of the query is rendered as a HTML page (thus explaining the <p/> in the middle of the query.

Mysql 5.1/5.5 and Unicode

I noticed that Mysql 5.1. and 5.5 has a difference with respect to the support for Unicode characters. Mysql 5.1 only supports the Basic Multilingual Plane (BMP), that is code points between 0 and 65535 (U+0000 to U+FFFF). Characters whose code points are greater than 65535 (U+FFFF) are called supplementary characters and support for these are included in Mysql 5.5. Read more at http://dev.mysql.com/doc/refman/5.5/en/charset-unicode.html.

Java String support both Basic Multilingual Plane and supplementary characters so this may cause some problems if a String is send directly into Mysql. Typically you will see this error message:
java.sql.SQLException: Incorrect string value: '\xC2\x9F' for column 'APN' at row 1


A little warning: even in Mysql 5.5 the "utf8" "charset" does NOT support supplementary characters. Use utf8mb4 instead.

07 February 2012

How to take a Java heap dump from command line

A heap dump is a great way to analyze a Java program in search of memory leaks. I mostly use Eclipse Memory analyzer for such an analysis. It is possible to acquire a heap dump from inside Eclipse Memory analyzer. But it may not always be possible to run a GUI in a foreign environment.

Use jps to find out the process id
C:\>jps
11448 MyLeakingApplication
12084 Jps

Use jmap to acquire the dump
C:\>jmap -dump:format=b,file=c:\Temp\mymemorydump.hprof 11448
Dumping heap to C:\Temp\mymemorydump.hprof ...
Heap dump file created