Drop By!

Found this interesting article Complement testing with code inspections which lists and explains 11 common mistakes.

The eSciDoc Infrastructure is published now under the CDDL-Licence .

eSciDoc uses the Fedora Repository as backend to store the digital objects and runs within a JBoss Application Server. Although it is still beta the main features are implemented and the install works without bigger problems now.

Hier einige freie Bücher von Galileo Computing auf deutsch.

Integrationshandbuch Microsoft-Netzwerk
Ubuntu GNU/Linux
C von A bis Z
Java ist auch eine Insel – umfangreich
Praxisbuch Objektorientierung – enthällt auch moderne Konzepte wie IoC und Dependency Injection (siehe Spring Framework)
JavaScript und Ajax – ist für Ende November versprochen

The Handle System provides an open source implementation of the Handle System protocol (read the license: 4263537/5024).

I was curious to see and learn from the code. Also I am developing webservices using Axis2 and Spring to wrap the Handles Client API. See also Adam Smith’s Article about Developing Handle System Web Services at Cornell University and the source code Cornell University published just yesterday – handle_services.

For that you need more information then provided by the written documentation. This kind of information you can find by studying the source code directly. Furthermore one also might wish to extend the provided resolution mechanism of the included proxy. So I decided to have the source code in the same workspace in Eclipse as my webservice project. How to set up an independend handle server at localhost to play with dummy handles for testing purposes I described here.

Download the hdl6.2.4.tar.gz file from Handle Software Download. From that archive you will need the src.zip (source code) und handle.jar files. Extract the src.zip file . From the handle.jar you will need to extract a library to compile the handle source:

• run jar to extract the package net.cnri: jar xvf handle.jar net/cnri
• build a new jar from the extracted files and folders: jar cvf cnriutil.jar net

You will also need the Java Servlet library ( jsdk.jar). Download it from here: SPECIFICATIONS – Download class files 2.3, rename the downloaded zip file to jsdk.jar.

How to import the code into a Eclipse Java project? There are two issues here – getting the libs right and the not eclipse-friendly project structure. This is the way it worked for me with Eclipse 3.2:

• Created a new, empty Project “hs2k” : File -> New Project , check “Use project folder as root for sources and class file”
• Imported the Handle source: Import -> General -> File System -> Browse to the hdl6.2.4/src folder and select all resources there, check “Create selected folders only”
• Configuring Build path: in Source-tab – set “Default Output Folder” to “hs2k/build”
• There were still errors because some libs are missing in the source code distribution.
• Getting the libs:
  • Created new folder “lib” under the projects root
  • Imported the cnriutil.jar and jsdk.jar into hs2k/lib folder
  • Adding the jars in lib to projects build path: Properties -> Java Build Path -> Libraries -> Add JARs

After that I had still some errors due to 6.0 compiler compliance level. Problem were variables called enum. Enum is a reserved Java keyword now. So I renamed the variables to enm.

There are two advices you can find in the build.xml and build.properties files – setting “maintainerMode = true” and override the build.properties by using user.properties. To use user.properties correctly I had to edit the build.xml file:

<!-- include user.properties file for custom config -->
<property file="user.properties">
<property file="build.properties">

The user.properties:

target = 1.6
# maintainerMode = true

I couldn’t use the maintainer Mode cause the appropriate root_info were not available in the source code. Still I could compile the source code by using the appropriate ant tasks in Eclipse and make server and clients run like described in this post.

… or rather compiled versus interpreted languages. If you use static final sometype varname in Java to define constants belonging to some class and you are using them in other classes you will have to recompile that classes too after changing the value of the constant.The constants get compiled into every class that use them. A problem which does not exist with interpreted languages. In Java you can use enums in some cases instead.

The Handle System provides a distributed infrastructure to manage persistent identifiers (PID) of digital ressources and resolve to their locations. There are several advantages of the Handle system like: scalability, detailed documentation, typed records structure, working global resolution infrastructure and last but not least – stable and readable Java code. By default a Handle server and its clients are designed to participate in the global PID resolution infrastructure.
But if you like to experiment with a local installation within your development environment you can follow these steps (you should be familiar with the Handle basics and Chapter 11 of the Technical Manual, also read the Handle System Service Agreement http://hdl.handle.net/4263537/5029 for more information).

Server setup:

1. Download the source of Handle software distribution from handle.net
2. This distribution comes with a SimpleSetup tool (see also the docs :Handle Manual), you have to edit some lines of the net.handle.server.SimpleSetup source file to be able to configure the server to run on 127.0.0.1 (you could do this by editing the config.dct file, but then there will be more work to set up the client). Change the getIPAddress() function to (commenting IP number check):
   

   /******************************************************************************
   * Get IP address for server
   */
   private static InetAddress getIPAddress()
   throws Exception
   {
   String localAddress = "",
   prompt = "Through what IP address will this server be accessible?";
   try {
   localAddress = InetAddress.getLocalHost().getHostAddress();
   }
   catch (Exception e)
   {
   localAddress = "";
   }
   if (localAddress.length() > 0)
   prompt = prompt + " [" + localAddress + "]";while (true)                                 // I.e., until "break" on success
   {
   String line = responseToPrompt(prompt);
   if ((line.equals("")) && (localAddress.length() > 0))
   line = localAddress;try {
   InetAddress listenAddr = InetAddress.getByName(line);
   /*  if (listenAddr.getHostAddress().startsWith("127."))
   throw new Exception("you must enter an address that is accessible from the network");*/
   return listenAddr;                                      // Success
   }
   catch (Exception e)
   {
   out.println("Invalid address (" + e + "), please try again.");
   }
   }
   }

3. Compile net.handle.server.SimpleSetup and run it. (I imported the source to Eclipse and run it directly from there)
4. Answer the questions as described in the manual. Use 127.0.0.1 as IP to run it locally.
5. Home a prefix of your choice by inserting it to the nas table (nas.jdb).
   1. Run the provided DBTool
      java -cp  ./bin/handle.jar net.handle.apps.db_tool.DBTool ./svr_1/
   2. Add a test naming authority like 0.NA/1 .
   3. Add a handle for the admin 1/ADMIN with HS_PUBKEY and HS_SECKEY type at index 300 and 301 (so you can experiment with both authentication methods):
      300 HS_PUBKEY 1110 <hexcode of your pubkey>
301 HS_SECKEY 1110 secret
      (I couldn’t get the variant using FILE <path>/svr_1/admpub.bin as value here to work, cause FILE contents is not read from the server prior to authentication, the path to admins pubkey assumes that servers working dir is “pathtohandleserver”/hs or set Java property -Duser.dir=”pathtohandleserver”/hs on start up)
   4. Create a few handles like 1/TEST with HS_ADMIN at index 100: use DBTools “Create Handle” -> Enter New Handle:”1/TEST” -> Ok -> Add -> Index: 100 , Type: HS_ADMIN , Data-Edit: select Admin Info and fill in Admin ID Handle: 1/ADMIN and Admin ID Index: 300 (or 301), set appropriate rights, add further values and types.
6. Edit the servers config.dct file:
   1. Add configuration options for independend server:
      

      "server_config" = {
      "server_admin_full_access" = "y"
      "allow_na_admins" = "no"

   2. Change admin values to
      

      "server_admins" = (
      "300:1/ADMIN"
      "301:1/ADMIN"
      )

      and same for other admin elements.
7. Start up server
   /hs$ java -cp ./bin/handle.jar net.handle.server.Main svr_1/

Configure Client (linux):

1. Create ~/.handle directory in users home.
2. $ cp  “pathothandleserver”/hs/svr_1/siteinfo.bin ~/.handle/resolver_site
3. In ~./handle create a file local_nas, which should contain one prefix on each line. Every request for a handle having a prefix contained in this file will be sent to the local resolution site. Add one line “O.NA/1″.
4. Restart server and browse the proxy at http://127.0.0.1:8000 .

Einen ausführlichen Artikel findest du hier: http://java.sun.com/docs/hotspot/gc5.0/gc_tuning_5.html und hier: http://java.sun.com/performance/reference/whitepapers/tuning.html . Siehe auch das Beispiel für Eclipse in Eclipse Crash Problem . Ein geeignetes Profiling-Tool ist GCViewer.

• Der Speicher für die JVM wird regelmäßig durch einen Garbage Collector gereinigt, um nicht benötigte Objekte zu entsorgen und Speicher wieder frei zu geben. (Das Ganze lässt sich vielleicht mit der Bettenverwaltung auf der Intensivstation einer Klinik vergleichen.)
• Es existieren mehrer GCen, die auf den jeweiligen Einsatzzweck hin optimiert sind.
• Die Arbeit des GC kostet Rechenzeit und ist deshalb mit Hilfe von Parametern einstellbar; kann sich als plötzliches “Hängen” der Anwendung bemerkbar machen.
• Der verwaltete Speicher wird eingeteilt in tatsächlich allozierten (also tatsächlich reservierten) und virtuellen Speicher (potentiell zur Nutzung vorgesehenen).
• Im Speicher der VM unterscheidet man 3 Haupt-Bereiche: Young Generation (kürzlich erzeugte Objekte, werden öfter eingesammelt vom GC, da die Erfahrung zeigt, dass diese auch schneller sterben), Tenured Generation und Permanent Generation (“Meta”-Bereich für Klassentemplates, Classloader, alles was die VM selbst benötigt um Aufgaben zu erfüllen, also Betten für die Nachtschwestern). Young und Tenure Generation gehören zum sogenannten Heap-Speicher, der der Anwendung selbst zur Verfügung steht.
• GC findet getrennt in YoungGen und TenureGen statt (Minor- bzw. Major-Collections), je größer diese Bereiche sind, desto seltener muss der GC nach toten Objekten suchen, um Speicherplatz frei zu geben.
• Die JVM kann beim Start mit Hilfe von Parametern “getuned” werden: http://java.sun.com/docs/hotspot/VMOptions.html
  • Aufruf: java -parameter anwendungs-class
  • -Xmx maximale Heapgröße (Virtuell+Reserviert), bsp. -Xmx1024m auf 1024 MB
  • Achtung! Diese Werte haben keinen Einfluss auf PermGen!
  • -Xms Mindest-Heapgröße (immer fest Reserviert)
  • -XX:MaxNewSize=64M max. Größe von YoungGen
  • -XX:NewSize=64M mind. Größe von YoungGen
  • -XX:MaxPermSize=128M max. Größe des PermGen Speichers, oft können auftretende Probleme hiermit gelöst werden, da dieser per default nur 64 M groß ist
• Für große Anwendungen reichen die default Einstellungen nicht aus.
• Für große Server:
  • Die default Einstellung (64M) ist oft viel zu klein. Besser soviel wie möglich Speicher zur Verfügung stellen.
  • Wenn man -Xms und -Xmx gleich setzt, erreicht man bessere Performance, da der Aufwand zur Allokation von zusätzlichem Speicher entfällt. Wenn dieser Wert aber unpassend gesetzt ist, kann die VM den Fehler nicht abfangen.
  • Speicher raufsetzten, wenn mehrere Prozessoren beteiligt sind, denn Speicherallokation kann parallelisiert werden.
  • Evntl. auch -XX:MaxNewSize und -XX:NewSize gleich setzen, aber nicht zu klein, bspw. wenn in einem Serveraufruf eine komplexe XML Datei verarbeitet wird oder, wie in Confluence möglich, eine große Excel-Datei geparst und importiert werden muss.
  • Da im Serverbetrieb viele Objekte nur einen Aufruf oder eine Session lang leben und der Lauf des GC im Unterschied zu einer GUI Anwendung von der Latenzzeit des Netwerks verdeckt wird, ist es evntl. sinnvoll (habe es nicht probiert) den YoungGen Bereich kleiner zu setzen (siehe -XX:NewRatio), wenn bekannt ist, dass die einzelnen Threads klein bleiben.
  • Für mehrer Prozessoren probiere den Throughput Collector (-XX:+UseParallelGC)

Weitere Tips (aus der oben genannten Quelle):

• For most applications the permanent generation is not relevant to garbage collector performance. However, some applications dynamically generate and load many classes. For instance, some implementations of JSPTM pages do this. If necessary, the maximum permanent generation size can be increased with MaxPermSize.
• Some applications interact with garbage collection by using finalization and weak/soft/phantom references. These features can create performance artifacts at the Java programming language level. An example of this is relying on finalization to close file descriptors, which makes an external resource (descriptors) dependent on garbage collection promptness. Relying on garbage collection to manage resources other than memory is almost always a bad idea.

Beachte auch (siehe http://java.sun.com/performance/reference/whitepapers/tuning.html)
:

Before you start to tune the command line arguments for Java be aware that Sun’s HotSpot™ Java Virtual Machine has incorporated technology to begin to tune itself. This smart tuning is referred to as Ergonomics. Most computers that have at least 2 CPU’s and at least 2 GB of physical memory are considered server-class machines which means that by default the settings are:

• The -server compiler
• The -XX:+UseParallelGC parallel (throughput) garbage collector
• The -Xms initial heap size is 1/64th of the machine’s physical memory
• The -Xmx maximum heap size is 1/4th of the machine’s physical memory (up to 1 GB max).

Ever wondered about regular “out of memory” crashes of Eclipse? One solution for you might be to set the Java Virtual Machine settings in case you use sun java (on linux)
eclipse -vmargs -Xms1024m -Xmx1024m  -XX:PermSize=256m  -XX:MaxPermSize=512m

(Thats rather pessimistic numbers, but my computer has 1,5 GB memory, so I can afford it.)

The problem seems to be that suns virtual machine is not reserving enough permanent generation space and can not handle big amounts of loaded classes. More detailed explanation you can find in my excerpt (in german language) Java Virtuell Machine Setzungen für große Serveranwendungen anpassen
“The permanent generation is used to hold reflective of the VM itself such as class objects and method objects. These reflective objects are allocated directly into the permanent generation, and it is sized independently from the other generations. Generally, sizing of this generation can be ignored because the default size is adequate. However, programs that load many classes may need a larger permanent generation.”( Frequently Asked Questions
about Garbage Collection
in the HotspotTM JavaTM Virtual Machine).
See also jmap – Memory Map, Eclipse Bug 92250.

A great explanation you can find at Frank Kieviet’s Blog:

• Classloader leaks: the dreaded “java.lang.OutOfMemoryError: PermGen space” exception
• How to fix the dreaded “java.lang.OutOfMemoryError: PermGen space” exception (classloader leaks)

More Virtuell Machine (Hot Spot) options at http://java.sun.com/javase/technologies/hotspot/vmoptions.jsp and tuning tips at http://java.sun.com/docs/hotspot/gc/.

Sometimes the management asks: Why do we need to hire Java programmers, can
we not let our scripters do the job?
Ok, you can answer this question by using a lot of buzzwords. But it seems to me to be more difficult to explain it to someone who has not at least a basic insight into the programming domain. Here is a (too?) simple java example:

/* straight forward coding you will see suprisingly often done by people with scripting background only, creates 1000 copies of "D" */
String[] a = new String[1000];
for (int i = 0; i < 1000; i++) {
a[i] = new String("D");
}

/* creates 1 instance of "D" and 1000 references to it,
in more serious operations that will have visible effect on the performance */
String[] a = new String[1000];
String d = "D";
for (int i = 0; i < 1000; i++) {
a[i] = d;
}

/* So what about this? */
String[] a = new String[1000];
for (int i = 0; i < 1000; i++) {
a[i] = "D";
}
/* Actually the compiler will treat "D" as a static symbol, no overhead is produced
*/

All literal strings and string-valued constants are kept in a pool maintained privately by the class String. You can use String.intern() to add you own strings to this pool. If the pool already contains a string equal to your String object, as determined by the equals(Object) method, then the string from the pool is returned, otherwise your String object is added to the pool and a reference to it is returned.

Check it out by yourself:

/**
* @see http://java.sun.com/docs/books/performance/1st_edition/html/JPRAMFootprint.fm.html
* @author Konstantin Rekk
*
*/
public class InitialisationIssues {  /**
* @param args
*/
public static void main(String[] args) {
// TODO Auto-generated method stubString[] a = new String[1000];System.out.println("before: " + getUsedMemory());
// String d = new String("-----DATA-----------------------------");
for (int i = 0; i <  1000; i++) {
// a[i] = new String("-----DATA-----------------------------"); ;
a[i] = "-----DATA-----------------------------";
}

System.out.println("after: " + getUsedMemory());

for (String el : a) {
System.out.println(el.toString());
}

}

private static long getUsedMemory() {
gc();
long totalMemory = Runtime.getRuntime().totalMemory();
gc();
long freeMemory = Runtime.getRuntime().freeMemory();
long usedMemory = totalMemory - freeMemory;
return usedMemory;
}

private static void gc() {
try {
System.gc();
Thread.sleep(100);
System.runFinalization();
Thread.sleep(100);
System.gc();
Thread.sleep(100);
System.runFinalization();
Thread.sleep(100);
} catch (Exception e) {
e.printStackTrace();
}
}

}

If you have a lot of eclipse workspaces to manage it is helpful to have them shown in the title bar of the window. Is there any setting in the preferences for this? I don’t know, but you can use the -showlocation option at start up:
eclipse  -showlocation -vmargs -Xmx700M