Dr. Dobb's Journal January, 2005
Jacl.NET is a port of the Jacl Java-based TCL interpreter to the .NET Framework. Jacl.NET gives .NET developers a TCL interpreter suitable for embedding within applications. In this article, I describe the issues I encountered in using Visual J# .NET to porting Jacl to .NET, and present an application of an embedded interpreter for database password management that eliminates the need to keep enterprise application passwords on disk. The complete source code for Jacl.NET is available at http://www.mailframe.net/ and from DDJ (see "Resource Center," page 5).
Jacl itself is a port of Sun's TCL interpreter from C to Java, combining core language and commands with Java-specific extensions. The port has existed in several revisions; the current version is 1.3.1 (http://sourceforge.net/projects/tcljava). However, unlike the C version of TCL, which supports a wide array of extensions such as OraTCL or Expect, Jacl addresses only the core TCL language. TCLJava, a second package intertwined with Jacl, provides the ability to script and control Java objects from TCL scripts running in the Jacl interpreter.
The Jacl port to J# .NET is intended to provide core TCL language scripting capability within .NET applications. As a scripting language, TCL can then be leveraged to add dynamic, interpreted functionality to .NET applications. The Jacl/TCLJava focus on allowing scripting of Java classes is removed and replaced with an emphasis of integration with the .NET environment, thereby leveraging the CLR and allowing multiple language development.
Porting Jacl to .NET lets you create TCL extensions using .NET, including making new TCL language commands in any .NET language. The port forms a platform to extend TCL for use within your own .NET applications as a command language, runtime expression interpreter, or script-based extensible application configuration system. Jacl in .NET provides anyone familiar with .NET and TCL the ability to extend the TCL language to suit their own application needs without the traditional C programming required to extend TCL.
The port began with the full Jacl 1.3.1 source tree taken directly from SourceForge. I created a J# project with the Jacl and TCLjava source directories included recursively to capture all Java language source files. I used J# to avoid porting the Java code to C#, making the best use of the existing Jacl code. The existing makefile was abandoned in favor of a .NET solution and project, since most .NET developers will use Visual Studio. The first pass of porting was simply getting the source to compile.
The first problem involved variable naming. Variables named enum are considered to be a keyword by the J# compiler in Visual Studio 2005. Prefixing these variables with the escape "@" character, turning them into @enum, lets them compile. Previous ports of Jacl using Visual Studio 2003 didn't require this, but in referring to the Visual Studio 2005 documentation for J#, the answer became clear—2005 adds enumerated type support to J# through the enum keyword.
The major effort in porting revolves around I/O processing. Versions of Jacl from 1.1.1 to 1.3.1 have different I/O abstractions. The most recent version (1.3.1) forms the major basis for the port making use of Java classes not available in .NET. The root of the porting problems occur in TclInputStream, which performs conversion with sun.io.ByteToCharConverter. Unfortunately, this class is not available in J#. Similarly, TclOutputStream performs conversion with sun.io.CharToByteConverter. Ultimately, the TclInputStream and TclOutputStream exist to have character and byte encoding streams available to the Channel class. This was introduced with Jacl 1.3.1 moving towards Java 1.2 capabilities. However, with J# at a level of Java substantially less than Java 1.2, and certainly not including Sun libraries, I merged the previous Jacl release implementation of Channel from the 1.1.1 code into the 1.3.1 code base.
This I/O port activity affects the FileChannel, StdChannel, and BgErrorMgr classes in the Jacl port for pure I/O. FblockedCmd and FConfigureCmd are affected to be compatible with the simplified I/O Channel from prior to 1.3.1 Jacl. For logical I/O, GetsCmd, OpenCmd, PutsCmd, ReadCmd, and SeekCmd are ported from 1.1.1 instead of from 1.3.1 to work with the back version Channel. The final piece of I/O, SocketCmd, is also ported from 1.1.1. In summary, Jacl.NET I/O is not up to the Jacl 1.3.1 level. Still, for my needs (and testing), there were no negative consequences.
Jacl provides the capability to call and script Java objects through TCLJava. While useful in a Java setting, this is less useful within a .NET context. Consequently, the commands and support classes (Java*) that form the Java language object scripting capability are excluded from the .NET port solution. This streamlines the port to focus on TCL functionality, substantially reducing the size of the Java-specific porting effort. (Replacing this with the ability to script any .NET object interactively is planned as an extension.)
Resource streams in Java aren't really supported in J# in the same fashion. I converted these to use .NET assembly resources. This is important because Jacl uses TCL scripts embedded as resources to configure itself on startup. The embedded resource is a startup script that registers commands for execution and defines commands as TCL functions. Using assembly resources is an example of mixing and matching .NET classes with Java core classes directly into J# source. Even though Java I/O and streams are available, I chose to use the .NET streams for reading a resource (see Listing One), which reads an embedded resource script that was compiled into the assembly into a string for runtime evaluation. Notice how you can get a Type from a J# instance exactly like a C# instance. Except that I compile it as J#, this piece of code looks like—and might as well be—C#. J# is close to C# in language in many cases, with the differences expressed in library support. By leveraging the .NET classes, these differences can be minimized. Finally, I removed the mapping for the Java command handler in the interpreter main object tcl.lang.Interp. This associated the command string jaclloadjava with a specific command object tcl.lang.JaclLoadJavaCmd.
These changes result in a core language TCL that compiles with J#. To fit in with the .NET console application model, I created a small console shell application that invokes the main interpreter class tcl.lang.Shell. Partly out of personal preference and partly out of wanting to demonstrate mixed-language development, this shell program (called "jaclsh") is created in C#. It is nearly a one-liner, directly invoking the Java-based main function passing the console arguments; see Listing Two. With the shell program in place, I ran the TCL test scripts included with Jacl. I removed the TCLJava-based tests—not supporting scripting Java objects, instead focusing on the tests for core TCL functionality.
In running tests, I found a series of small dependencies on the Java platform separate from the Java language—issues about paths, application names, and host system information. This led to changes in tcl.langInfoCmd, reminding me that Java is more than a language—it's also a platform. This is important in porting. There are issues with any port that are larger than just the code in the program, and you need to keep in mind how the code touches the environment and what the assumptions are for that code in its native environment. I had similar problems with the clock command because it depends on the environment variable system in Java. For instance, formatting time zones as GMT+00:00 instead of the expected GMT, as .NET shows time zones relative to GMT in string formats, instead of three-character time zone name abbreviations. Consequently, I completely replaced the clock command, converting it to use .NET-style dates, including System.DateTime parsing and formatting, replacing the custom date parsing, and processing in TCL. This was more to my liking, having become much more of a .NET programmer than a Java programmer. This required that I expand support to include long integers to hold the nanosecond scale-tick values that System.DateTime uses, an unexpected piece of additional work, but beneficial for high-precision clock times.
With a working .NET port, my next step was to extend the interpreter to provide custom commands. Essentially, all extensions to Jacl take the form of commands. TCL has a simplified, regular grammar with a mapping of a command word to a command class. Each command class is instantiated as an object and then takes an array of parameters, which may be literal strings or numbers, or the evaluated results of other commands. The commands themselves follow the classic command pattern, which creates an instance of a command class to serve as a verb or function, then invokes operations with an execute operation when you run a script. In the case of Jacl, the classes implement tcl.lang .Command, and the execute operation is called cmdProc. This object as function metaphor is prevalent in Jacl.
Major enterprise applications—database servers or even Windows itself—have remote command-line interfaces, such as isql for SQL Server, or netsh for Windows (among others). You can use Jacl.NET to easily create your own command-line control interface.
Practically every enterprise application accesses a database, and most of those store the connection information in either the registry or a configuration file. Many will even store the password in plain view on disk. Using Jacl.NET, a custom command, and an interpreter instance mounted with .NET remoting, you can create a means to configure applications at runtime without fixed configuration files left on the server. This lets you authenticate an enterprise server application to a database without having the password be part of the application itself or in any configuration file. As an example of command-line control, I have built a logon command (Listing Three) that tells a remote-process-running server where to connect for database access.
Creating a custom command is a straightforward implementation of the tcl.lang.Command interface. Jacl parses the arguments and parameters for you, passing a simple array of object parameters with the zero position being the name of the command. For logon, a single parameter connection_string suffices. It checks for the right number of parameters, and if they are in place, a connection to a database server is opened with the passed connection string parameter, and finally the connection string is saved in a static variable making it visible to the rest of the application. The command extension is implemented in C# to demonstrate mixed-language development. This has a C# class implementing an interface originally authored in Java. The setResult call is the output mechanism to allow a command to return information to the caller, as the cmdProc method itself has a void return type.
With the command implemented, the next step is to make it accessible. This access is provided by a simple remoting server exposing the interpreter on a socket. While this is not the most secure option (remoting lacks transport encryption), it illustrates the point. In a secure setting, you could use an SSL socket for remoting or WSE 2.0 with WS-Security encryption to provide transport and some form of authentication and access control. However, remoting makes a simpler and shorter illustration of the remote scripting principle without including a lot of plumbing to expose the interpreter on the network.
InterpreterService registers the new custom command with a string that is the actual command text to type in TCL scripts in an interpreter instance; in this case, mapping the command string logon to the class SampleServer.LogonCommand via the Extension class method loadOnDemand. Extension serves as a registration hub for installing commands into an interpreter instance. The TCL interpreter uses the word to command mappings to locate and invoke command objects based on the passed input script.
The interpreter instance in this remote server is accessed by a single remote method that calls eval to run the script text. A class inheriting from MarshalByRefObject wraps the interpreter and provides an input method to submit scripts. On a successful run, the output of the script run is returned as a string. Exceptions are propagated automatically by remoting as long as they are serializable!
The service is exposed on a TcpChannel as a well-known object allowing lookup by URL. The client uses a remoting URL to find a server to run scripts; see Listing Four.
The remote client is straightforward, connecting to a published remote instance of the InterpreterService before invoking a script passed as a command-line argument. I implemented this client in VB.NET, continuing the theme of mixed-language development. The client takes a script passed as a command-line argument, invokes the remote interpreter, and then echoes the response string to the console (Listing Five).
This client could be extended to read a file full of script commands, passing the entire content. Another option would be to create a remote TCL command that takes a set of scripts as a TCL argument in a client-side interpreter, passing the scripts to a server-side interpreter. This lets you use the Jacl shell as the local client interface instead of a single command-line call. Having a local shell lets you use TCL scripts to coordinate multiple commands from the client to a single server, or send commands to multiple servers each exposing the remote interpreter service.
To invoke Jacl.NET, you first run the remote interpreter server; for example:
SampleServer.exe
press enter to exit...
With the remote interpreter running, you issue a command at the prompt to check the error handling:
SampleClient.exe
"logon {bad connection string}"
In this example, the format of the initialization string does not conform to specification starting at index 0. Then you actually connect:
SampleClient.exe
"logon {Server=DOOZEY;user=
XXX;password=XXX;Database=XXX}"
which echoes back the success message:
Connected
This simple scenario illustrates all the basic principles in creating your own commands—implement the tcl.lang .Command interface, create an Interp, register with Extension, and make it available to call.
This example also illustrates the point of remotely altering the running state of a server with script. Using such a system in practice lets you control a custom enterprise server with an extensible mechanism that leverages an existing, well-known language. The benefit here is in control. I have used this technique with enterprise integration projects to get fine-grained control of the startup sequence of multiple interoperating services and to remotely manage clusters of similar servers with predefined scripts. Allowing the remote reconfiguration of servers at runtime is a way to centralize and control configuration and is easier than trying to keep hundreds of configuration files in sync across a server farm.
There are a number of improvements that can be made to Jacl.NET. For instance, adding .NET language object scripting capabilities, much like TCLJava, would be useful, allowing interpreted prototyping similar to other scripting projects such as IronPython (http://www.ironpython.com/). Such commands could be effectively implemented with reflection. This feature is a planned extension, based on demand. To date, the primary use of Jacl.NET has been for embedding within applications, not scripting objects.
Sticking with configuration and control, more than just database connections can be configured. Keeping a simple configuration class with dynamic settings, for example adding a hash table to the logon command to allow storing more than one setting, enables a system of hot configuration changes, letting you access and modify running server settings without a restart to pick up an application configuration file.
TCL in your application can give you capability for evaluated, user-defined fields, and formulas or customizable event-handling scripts. Lots of little pieces of math can be delegated and interpreted easily and efficiently to make fairly sophisticated algebraic calculators within your programs.
The command-line interpreter in Jacl forms the basis for creating your own command-line shell programs with a console user interface. This is an area where .NET is lacking. With all its emphasis on graphical and web UIs, the built-in shell processing is essentially limited to reading/writing lines of text. By implementing commands wrapping your functionality, you can make it all available at the command line, complete with a line buffer and inline editing similar to cmd. Overall, Jacl.NET is a useful tool to plug gaps in .NET with dynamic scripting capability.
DDJ
try
{
System.IO.StreamReader rdr = new System.IO.StreamReader(
this.GetType().get_Assembly().GetManifestResourceStream(resName));
eval(rdr.ReadToEnd(), 0);
} catch (Exception e2) {
throw new TclException(this, "cannot read resource \"" + resName + "\"");
}
Back to articlestatic void Main(string[] args)
{
try
{
tcl.lang.Shell.main(args);
}
catch (Exception ex)
{
System.Console.Error.WriteLine(ex);
}
}
Back to articlepublic class LogonCommand : tcl.lang.Command
{
/// <summary>
/// This holds the connection string once a connection is made.
/// </summary>
public static string ConnectionString;
#region Command Members
public void cmdProc(tcl.lang.Interp interp, tcl.lang.TclObject[] objv)
{
//check argument length
if (objv.Length == 2) //enough command arguments
{
try
{
System.Data.SqlClient.SqlConnection conn =
new System.Data.SqlClient.SqlConnection(objv[1].ToString());
conn.Open();
ConnectionString = conn.ConnectionString;
conn.Close();
interp.setResult("Connected");
}
catch (Exception ex)
{
interp.setResult(ex.Message);
}
}
else
{
//standard message for missing arguments
throw new tcl.lang.TclNumArgsException(interp, 1,
objv, "connection_string");
}
}
#endregion
}
Back to articleclass Program
{
static void Main(string[] args)
{
InterpreterService service = new InterpreterService();
TcpChannel channel = new TcpChannel(55000);
ChannelServices.RegisterChannel(channel);
ObjRef mounted = RemotingServices.Marshal(service,"interpreter");
System.Console.WriteLine("press enter to exit...");
System.Console.ReadLine();
}
}
/// <summary>
/// Expose the <see cref="tcl.lang.Interp"/> via remoting.
/// </summary>
public class InterpreterService : MarshalByRefObject
{
private tcl.lang.Interp _interpreter = new tcl.lang.Interp();
public InterpreterService()
{
tcl.lang.Extension.loadOnDemand(_interpreter, "logon",
"SampleServer.LogonCommand");
}
/// <summary>
/// Execute a script, returning the evaluated results.
/// </summary>
/// <param name="script"></param>
/// <returns></returns>
public string RunScript(string script)
{
try
{
_interpreter.eval(script);
return _interpreter.getResult().ToString();
}
catch (Exception ex)
{
return ex.ToString();
}
}
}
Back to articleModule ClientEntryPoint
Sub Main(ByVal args As String())
If (args.Length > 0) Then
Dim chan As New TcpChannel()
ChannelServices.RegisterChannel(chan)
RemotingConfiguration.RegisterWellKnownClientType(GetType
(SampleServer.InterpreterService),
"tcp://localhost:55000/interpreter")
Dim remote As New SampleServer.InterpreterService()
System.Console.WriteLine(remote.RunScript(args(0)))
End If
End Sub
End Module
Back to article