This document outlines the features and major design points of the Layout Runtime prototype. The prototype is currently under development and is subject to change.
The goal of project Panama is to create a standard way to interop with native languages and libraries. Layouts are a key feature of project Panama, they represent how data structures are arranged in memory in the form of facades. Facades are java interfaces that merely represent the underlying data. They provide accessors to read and modify the data. The major components for layouts are:
The Layout Descriptor Language (LDL) is a language that describes how native data is laid out and how to access it (more details here). A Layout Description (LD) contains a description of native data structure (written in LDL). Appendix A contains the LDL grammar for the current prototype.
The groveller is a parser that reads in native structure definitions and outputs an LD file. The current prototype does not contain a groveller component but there are some ongoing projects to implement a C/C++ groveller.
com.ibm.parser.LD2J.Parser
Parser [inputFile] [outputDir] [packagePath]
Arguments must be in order.
inputFile
The path of the file that contains the Layout Descriptions
outputDir
The directory where the java facades will be generated
packagePath
The default package path for the java facades. Note this is an optional feature.
LD2J is a parser written in Java that reads in a file containing Layout Descriptions and outputs java facades. A facade is created for each structure defined in the LD file.
java com.ibm.parser.LD2J.Parser /com.ibm.layouts/user/com/user/examples/myline/LD com.ibm.layouts/user/com/user/examples/myline/ com.user.examples.myline
A generated facade would look like the following:
package com.user.examples.myline;
import com.ibm.layout.Layout;
import com.ibm.layout.LayoutDesc;
import com.ibm.layout.LayoutType;
/*
* Licensed Materials - Property of IBM,
* (c) Copyright IBM Corp. 2014 All Rights Reserved
*/
@LayoutDesc({"start:Point:8","end:Point:8"})
public interface Line extends Layout {
public long sizeof();
public abstract Point start();
public abstract Point end();
@Override
public String toString();
}
The top level type is LayoutType, all layout types inherit from this. LayoutType contains factory methods for instanciating primitive layout arrays, as well as binding a layout to a Location.
getPrimArray1D(Class<?>, long)
getPrimArray2D(Class<?>, long, long)
getPrimUserArray1D(Class<AE>, Class<?>, long)
bindLocation(Location)
The Layout interface represents a singleton layout. This type defines a complex group of contiguous fields that can be accessed by a single memory address and an offset. All generated facades inherit from Layout. The Layout interface adds the following method:
getLayout(Class<T>)
Layout instanciation example
A layout can be created the following way:
Line l = Layout.getLayout(Line.class);
//Line is a facade
//l is a runtime instance of the facade
User Defined Layout Example
A user can subclass the generated facades to add additional functionality.
//Line is a facade
//Additional functionality is added in MyLine
public interface MyLine extends Line {
//default methods to add functionality
public default double length() {
int xdiff = end().x() - start().x();
int ydiff = end().y() - start().y();
return Math.sqrt(xdiff * xdiff + ydiff * ydiff);
}
public default void st(int x, int y) {
start().x(x);
start().y(y);
}
public default void en(int x, int y) {
end().x(x);
end().y(y);
}
}
MyLine ml = Layout.getLayout(MyLine.class);
The Array1D and Array2D interface provide an access pattern for uniform fixed-length repeating layout types. Array1D defines a single dimension of repeating layout types where each layout can be accessed with an array base address and an offset calculated by multiplying its size by its index. Array1D contains the following methods:
getArray1D(Class<T>, long)
getUserArray1D(Class$ltAE>, Class<E>, long)
at(long)
getLength()
put(long, T)
Array2D defines a two dimensional grid of layouts where each layout can be accessed in row-major order. Each row of the array is the same size.
getArray2D(Class
at(long, long)
dim1()
dim2()
put(long, long, T)
Array Example
//creates a 1D layout array of 10 Points
Array1D<Point> pts = Array1D.getArray1D(Point.class, 10);
//Point is a facade
//pts is a generated instance of an Array1D
The BooleanArray1D ... ShortArray2D are the built-in layout interfaces used to create array layouts of primitive java types. This is a work around created in an attempt to avoid wrapping primitives in Layouts, it might be obsoleted by generics over primitives. The following shows how they can be used.
ByteArray1D b = LayoutType.getPrimArray1D(byte.class, 10);
Ideally "getPrimArray1D(byte.class, ..." would be pushed down into [Byte, Int, ...]Array1D byte.class. But, the current protoype has one common factory method for primitive arrays so it is placed in the common super type.
Runtime classes of a Layout type are created by the LayoutFactory implemented in LayoutHelper. This LayoutHelper class creates all runtime classes in a separate classLoader (LayoutHelper.ImplClassLoader) in the "com.ibm.layout.gen" package directory. The reason behind this is to protect the generated classes from naming conflicts as well as to ensure that their fields cannot be reflected. The following shows the call hierarchy for loadLayoutClass:
com.ibm.layout.Array1D.getArray1D(Class<T>, long)//instanciate 1D array
com.ibm.layout.Array2D.getArray2D(Class<T>, long, long)//instanciate for 2D array
com.ibm.layout.Layout.getLayout(Class<T>)//instanciate layout
com.ibm.layout.LayoutHelper.genLayoutImpl(Class<T>)//initializes static fields in the
com.ibm.layout.LayoutHelper.ImplClassLoader.loadLayoutClass(Class<? extends Layout>)//generates runtime layout class
The loadLayoutClass method generates the runtime class for the specified layout interface (Class<? extends Layout> layoutInterface) in the following steps
Instances of facades have the following naming convention, "[LayoutName]Impl". For Layout arrays they are "[ElementLayoutName]1DImpl".
Bytecode generation is done using the ASM framework. There is no bytecode generation for java primitive arrays, those are loaded from class files stored in the "com.ibm.layout.gen" package.
Each layout instance is composed of an immutable Location object that represents the addressable memory range. In this current prototype it is composed of two fields, an object and a long.
To use a layout, it must be bound to a location using "bindLocation(Location loc)". The Location class has the following constructors:
Location(byte[])
Location(Location, long)
Location(long)
Creating a location as well as binding a location will require security checks to ensure that the layout fits in the location it is bound too and that the location is in a valid memory region. The specific details regarding how 'valid memory region' is defined is one of our main design discussion points that will be addressed in later versions.
The LDL grammar used in the current prototype is as follows:
"Qualified Name" { ',' "[Field Name][Type]{'['Number of Elements']'} ':' (Size | 'pointer' | 'Layout' Name)"}
where:
Basic Example
The following struct "Point2D"
struct Point {
uint32_t x;
uint32_t y;
}
Would have the corresponding LD:
"Point", "x:jint:32", "y:jint:32"
LD Array Example
One can define a layout with array fields by using the following notation.
struct A {
uint32_t x;
uint32_t y[10]; //this is an array of fields
}
LD: "A", "x:jint:4", "y[10]:jint:40"
Nested fields
This example shows how to describe nested structures
struct Line {
Point start;
Point end;
}
LD: "Line", "start:Point:8", "end:Point:8"
Note: The grammar used in the prototype is not the latest version described in the State of the LDL document. This grammar will be updated in future versions of the prototype.