--- title: How to use IntradayLoggerFactory in a development environment --- The `IntradayLoggerFactory` class is a Deephaven component used internally by the `generate_loggers` script. It is responsible for parsing table schemas to generate logger code for Java and C# loggers, and Java listener code for schemata that have Logger, Listener, or LoggerListener sections defined. These classes are used to write and read data in Deephaven binary log files, predominantly as part of data streaming. Deephaven data streaming in general is covered in the [Streaming Data](../data-guide/streaming/streaming-intro.md) overview. Using `IntradayLoggerFactory` in a development environment, typically with an IDE such as IntelliJ IDEA, makes it easier to develop and test the schema and logging process for a new data source. The other alternative is to deploy the schema to a Deephaven server, along with any dependency JARs, then run `generate_loggers`, and then copy the resultant JAR or classes back to the development system. Clearly, generating logger and listener classes locally on the development system is easier and faster and allows a more seamless integration with the development process. ## The example project The [deephaven-intraday-logger-factory example project](https://github.com/deephaven-examples/deephaven-intraday-logger-factory) contains example pieces and tasks to demonstrate Java logger class and application creation. Clone or download the repository to use it locally. ![The `SampleLogger` folder expanded to show contents](../assets/tips-and-tricks/intradaylogger1.png) The project includes the following components: - `SampleLogger.TickData.schema` - This is a schema file for a simple table with a LoggerListener section. - `TickDataLogFormat1Interface` - This is a Java interface class that defines the properties to be logged by the `TickData` schema. - `com.samplelogger.TestLog` - This is a logger application that uses the logger class generated from the schema to write some sample data to a binary log file. - `dependencies.gradle` - A Gradle file that defines the base Deephaven DB module dependency to be downloaded from Artifactory. - Gradle tasks (under the `deephaven` group): - `compileInterface` - This compiles the `TickDataLogFormat1Interface` class. This is a prerequisite to building or running any other components in the project. - `intradayLoggerFactory` - This executes `IntradayLoggerFactory` to read the schema and generate the logger and listener classes. - `runTestLogger` - This runs the example logger application, which will use the generated logger to write two records to a binary log file. - Run configurations (duplicates of Gradle tasks, to show another way the classes can be used): - `Generate LoggerListener` - This executes `IntradayLoggerFactory` to read the schema file and generate the logger and listener classes. - `Run Logger` - This runs the example logger application, which will use the generated logger to write two records to a binary log file. ![The `ILF-Gradle` project components](../assets/tips-and-tricks/intradaylogger2.png) In addition, the Gradle configuration for the project extends clean so that clean will also delete the generated logger and listener classes, so the clean task will restore the project to its initial state. To use this project, it will likely be necessary to edit the `dependencies.gradle` file to change the Deephaven version. `repoUser` and `repoPassword` must be set in `gradle.properties`. ```groovy repositories { mavenCentral() maven { name = 'Release Artifacts' url = uri('https://repo.deephaven.io/maven/release') credentials { username = findProperty("repoUser") ?: "" password = findProperty("repoPassword") ?: "" } authentication { basic(BasicAuthentication) } } } apply plugin: 'java' apply plugin: 'base' dependencies { version = '1.20240517.+' def fishlibVersion = '25.20240517.+' implementation 'iris:DB:' + version implementation 'iris:configs:' + version implementation 'fishlib:Core:' + fishlibVersion implementation 'fishlib:Internals:' + fishlibVersion implementation 'fishlib:FishHash:' + fishlibVersion testImplementation platform('org.junit:junit-bom:5.10.0') testImplementation 'org.junit.jupiter:junit-jupiter' } ``` ## Use of IntradayLoggerFactory `com.illumon.iris.db.tables.dataimport.IntradayLoggerFactory` is executed from its `main` method. It takes no program arguments, but its behavior can be modified by setting JVM properties. If no special properties are set, it will expect to read all schemata from a default location and create loggers and listeners for all of them. In the example project, properties are set to limit it to processing a single local path for the schema and generating classes under the `SampleLogger/src/main/java/` path. The base directory (`SampleLogger`) is set via the `IntradayLoggerFactory.javaGenerationDirectory` property, while the subdirectory under it (`/src/main/java/` by default) is controlled by the `IntradayLoggerFactory.javaDir` property. These JVM properties are documented in the `build.gradle` file: ```groovy skip-test // iris-defaults.prop and needed jars come from the iris:DB entry in dependencies.gradle systemProperty 'Configuration.rootFile', 'iris-defaults.prop' systemProperty 'devroot', "$rootDir" // use local schema files instead of the schema service (consideration for v1.20200928 and later) systemProperty 'SchemaService.defaultImpl','com.illumon.iris.db.schema.LocalTransientSchemaService' systemProperty 'SchemaService.Local.sourceDirectory','SampleLogger/schema/' systemProperty 'SchemaConfig.resourcePath.irisInternal','SampleLogger/schema/' // use local prop files instead of the configuration service (consideration for North Arapaho and later) systemProperty 'com.fishlib.configuration.PropertyInputStreamLoader.override','com.fishlib.configuration.PropertyInputStreamLoaderTraditional' // tee log messages to stdout systemProperty 'LoggerFactory.teeOutput','true' // set the directory name into which logger and listener classes should be generated systemProperty 'IntradayLoggerFactory.javaGenerationDirectory','SampleLogger' ``` To use `IntradayLoggerFactory` for a different schema, change `SchemaService.Local.sourceDirectory` and `SchemaConfig.resourcePath.irisInternal` (similar properties with applicability to newer or older versions of Deephaven, respectively) to "point" to the location of the schema files. Note that all classes referenced by the schema, including logger interfaces, must be available in the classpath and compiled as a prerequisite to running `IntradayLoggerFactory`. ## Use of the examples in the project Running the `deephaven/runTestLogger` Gradle task runs all required tasks needed to get from a schema and logger application to actually logging data: 1. Compile the `TickDataLogFormat1Interface` 2. Run the `intradayLoggerFactory` task to generate the `com.samplelogger.tickdata.gen.TickDataLogFormat1Logger` class (under `SampleLogger/src/main/java`) 3. Compile the `com.samplelogger.tickdata.gen.TickDataLogFormat1Logger` class 4. Build and execute the `com.samplelogger.TestLog` class, which will then create a new binary log file and write two rows to it: ``` Timestamp,Exchange,Security,Underlying,BidSize,BidPrice,AskSize,AskPrice,TradeStatus,TickCondition,ExchangeTimestamp,ExchangeTimeNanos 1576017365892,N,"AAPL_12321","AAPL",10,777.77002,10,888.880005,N,A,1000000000,1000000000000000 1576017365892,A,"GOOG_12321","GOOG",10,777.77002,10,888.880005,F,N,1576017365892,1576017365892000000 ``` This log file is generated with a file name using the following syntax: `TickDataLog.bin.yyyy-MM-dd.hhmmss.nnn` For example: `TickDataLog.bin.2019-12-10.153605.822-0700` In newer releases (v1.20190607 and later) the default path for the file is `/var/log/deephaven/misc`. In earlier releases, the local `SampleLogger/working_directory_for_older_versions/logs` directory will be used. The file name is controlled by the call to it in `TestLog.class`. For newer versions, you can change the log path by setting the JVM property `logroot`, but the file will still be created under the misc directory under the defined `logroot`. You can run individual Gradle tasks separately. When executing them from the IntelliJ IDEA run configurations, be aware that there is no chaining involved. Therefore, you need to compile the logger interface explicitly first. To do this, right-click on the interface and select "Recompile." Additionally, you will need to run `IntradayLoggerFactory` to generate the logger class before you can execute the logger application itself. To run `IntradayLoggerFactory` from the command line, you need a set of Deephaven JARs in a location where you can pass them to a classpath. Gradle downloads JARs to GUID-named cache directories that are not very accessible for non-Gradle command line use. You can use the Deephaven Launcher or the Configuration Updater to synchronize needed JARs and other resources from a Deephaven server to a client/development system. ### Example A sample `IntradayLoggerFactory` execution command line follows: ```bash java -cp ~/Downloads/IllumonLauncher/TPlus/java_lib/*:~/Downloads/IllumonLauncher/TPlus/resources/:./SampleLogger/build/classes/java/main \ -server -Xmx4096m \ --add-opens=java.base/java.lang=ALL-UNNAMED \ --add-opens=java.base/java.nio=ALL-UNNAMED \ -DConfiguration.rootFile=iris-defaults.prop \ -DIntradayLoggerFactory.javaGenerationDirectory=SampleLogger \ -DSchemaService.defaultImpl=com.illumon.iris.db.schema.LocalTransientSchemaService \ -Dcom.fishlib.configuration.PropertyInputStreamLoader.override=com.fishlib.configuration.PropertyInputStreamLoaderTraditional \ -DSchemaService.Local.sourceDirectory=SampleLogger/schema/ \ -DSchemaConfig.resourcePath.irisInternal=SampleLogger/schema/ \ -DLoggerFactory.teeOutput=true \ com.illumon.iris.db.tables.dataimport.IntradayLoggerFactory ``` The classpath references the `java_lib` directory and resources from a Deephaven instance (in this case, the instance is called TPlus). The classpath also references `SampleLogger/build/classes/java/main` because this is where the class for the logger interface can be obtained. `-D` is used to pass other JVM arguments needed to set `IntradayLoggerFactory` to read a single, local, schema directory, and to direct it to create generated classes under `SampleLogger`. ## Related documentation - [Schema overview](../data-guide/tables-and-schemas.md) - [Streaming data](../data-guide/streaming/streaming-intro.md)