Mapping data module for aSISt system
Kunde: Die Benutzer des aSISt-Systems
Requirement
Flexible tool providing functionality of declarative import for aSISt’s various reports types. Defining expressions to count reported items which maps source data to destination report’s cells. Independence of source data format for given report type. Fully operative on any table-formated data having no assumptions about source schema. Support for CSV and Excel 97 format. Import to tables with fixed as long as dynamic structure (tuple tables and typed dimensions). Import process based on terse and expressive mapping language. Defining mapping expressions set in reports domain. Management and store mechanism for prepared mapping expressions. Support for archive, export and import actions for mapping expressions.
Solutions
The biggest challenge was to design flexible language giving a way to precisely point out required source data for each destination report’s cell. It had to have an opportunity to do custom counting during import process (arithmetical operations, aggregates). It had to be easy and characterized with simple notation to provide easy way for not advanced users to map simple data sets without full understanding of its grammar. On the other hand it had to provide a way for advanced user to handle more sophisticated cases giving required expressiveness.
Moreover it was essential to support import for tables with dynamic structure where it is impossible to assume amount of data at mapping definition level. To provide flexible mechanism to import this sort of data an extension of mapping language was introduced. It gives an extra grouping tool for this types of mappings.
It was important to handle large data sets within reasonable processing time. Thanks to employing known from stream data management systems concept of processing queries on the fly, it was possible to reduce memory usage as well as processing time. The amount of memory usage during import process depends on data and complexity of expressions but is much lower than data sets itself. Additionally module tries to do some processing simultaneously which appears with significant speed up at dual-core processors.
Form user’s point of view it was essential to place edition of mapping expressions in report’s domain. To facilitate its understanding, mapping editor was introduced which has report’s structure and user edits expression within report’s cells.
Mapping module gives a possibility to process data which has partially prepared structure for not advanced user and also it can be used to process more complex expressions on large data sets with custom structure. The process can be done on average computers within reasonable time in both cases.
Technology
Application was coded in Java 6. Spring Framework was employed as an lightweight application container. It supports Oracle 9i and DerbyDB data base systems. Hiberate was used as object-relational mapping tool. ANTLR library was used for parsing mapping expressions.