Chapter 7  For Galax Developers

7.1  Galax Source Code Architecture

The Galax source-code directories roughly correspond to each phase of the query processor. (Put link to Jerome’s tutorial presentation here)

The processing phases are:

Document processing

  Document Parsing =>  
 [Schema Normalization (below) =>]
    Validation => 
      Loading => 
        Evaluation (below)

Schema processing

  Schema Parsing => 
    Schema Normalization =>
      Validation (above)
      Static Typing (below)

Query processing

  Query Parsing => 
    Normalization => 
   [Schema Normalization (above) =>]
      Static Typing (optional phase) => 
        Rewriting => 
          Compilation => 
         [Loading (above) =>]
            Evaluation =>
              Serialization 

7.1.1  General

Makefile

• Main Makefile

base/

• Command-line argument parsing
• Global variables (conf.mlp)
• XQuery Errors
• String pools
• XML Whitespace handling

ast/

• All ASTs: XQuery User & Core, XQuery Type User & Core
• Pretty printers for all ASTs

config/

monitor/

• CPU &/or memory monitoring of each processing phase

toplevel/

• Main programs for command-line tools (see ’Generated executables’ below)

website/

• Local copy of Galax web site

7.1.2  Datamodel

datatypes/ (*** Doug)

• XML Schema simple datatypes – Lexers and basic operations
• datatypes_lexer.mll To learn about O’Caml lex, read: http://caml.inria.fr/ocaml/htmlman/manual026.html Sections 12.1 and 12.2 Other examples of lexers in lexing/*.mll

  We are going to extend this module to include lexer for: xsd:date, xsd:time, xsd:dateTime, xs:yearMonthDuration, xs:dayTimeDuration (Skip Gregorian types for now, xsd:gDay, xsd:gMonth, etc)

• dateTime.ml,mli This module will implement the datatypes and basic operations

namespace/

• XML Qualified Names (prefix:localname) – Lexer and basic operations – QName resolution prefix => URI
• Names of builtin functions & operators

dm/ (*** Doug)

• Abstract data model interface for Nodes
• Concrete data model implementation for AtomicValues

datamodel/

• Main-memory implementation of abstract data model for Nodes (  Document-Object Model or DOM)

jungledm/

• Secondary storage implementation of Galax datamodel (Jungle)

physicaldm/

• Physical data model

streaming/

• XML parser to untyped and typed SAX streams
• export datamodel to SAX stream

7.1.3  Processing Model

procctxt/

• Processing context contains all query-processor state:
  • Parse context
  • Normalization context
  • Static context
  • Rewrite context
  • Dynamic context

procmod/

• Processing model dynamically "glues" together phases (controlled by command-line arguments or API)

7.1.4  Query Parsing

lexing/

• Lexers for XQuery (excludes all simple datatypes)

parsing/

• Parsing context
• Parsing phase

7.1.5  Normalization

normalization/

• Normalization context
• Normalization phase (XQuery AST => XQuery Core AST)
• Overloaded functions

7.1.6  Static Typing

fsa/

• Finite-state Automata for checking sub-typing relation

typing/

• Static-typing context
• Static-typing phase

7.1.7  Schema/Validation

schema/

• Schema-validation context
• Schema normalization phase (XML Schema => XQuery Core Types)
• Document validation phase
• Judgments(functions) for comparing XQuery types

7.1.8  Rewriting

cleaning/

• Logical optimization/rewriting phase
• Sort-by-document order (DDO) optimization

rewriting/

• Generic AST rewriter

7.1.9  Compilation

compile/

• Compilation phase

algebra/

• AST for compiled algebra
• Dynamic context
• Implementations (dynamic) of most built-in functions & operators

7.1.10  Evaluation

evaluation/

• Evaluation phase

stdlib/

• Static typing of built-in functions & operators
• Implementations (dynamic) of built-in functions fn:doc, fn:error
• Signatures of built-in functions & operators (pervasive.xqp) Corresponds to sections in http://www.w3c.org/TR/xpath-functions/

7.1.11  Serialization

serialization/

• Serialize SAX stream to XML document (in O’Caml formatter)

7.1.12  Testing

usecases/

• Tests of XQuery Usecases Implements examples in http://www.w3.org/TR/xquery-use-cases/

examples/

• Tests of O’Caml, C & Java APIs

regress/

• Regression tests (needs separate xqueryunit/ CVS package)

7.1.13  APIs

galapi/

• O’Caml, C & Java APIs to Galax processor

7.1.14  External libraries & tools

tools/

Required tools:

• http
• pcre
• pxp-engine
• netstring

Optional supported tools:

• Jungle

Optional unsupported tools:

• glx_curl
• jabber

7.1.15  Extensions

extensions/

• apache
• jabber

7.1.16  Experimental Galax extensions

projection/

• Document projection

wsdl/

wsdl_usecases/

• Web-service interfaces

7.1.17  Documentation

• Changes Change log!! Protocol: always document your changes in Changes file; use log entry as input message to ’cvs commit’
• BUGS Out of date
• LICENSE
• README
• STATUS
• TODO

7.1.18  Generated executables

ocaml-galax

O’Caml top-level interpretor that loads Galax library. Usage: ocaml-galax -I $(HOME)/Galax/lib/caml-devel

galax-run

Complete XQuery engine For Usage: galax-run –help See also: all: rule in usecases/Makefile

galax.a

Library versions of Galax

galax.cma

byte code

galax.cmxa

machine code

galax-parse

Syntax checking on query, validation on a document

galax-compile

Parsing, normalization, optimization, and prints resulting expression

galax-mapschema

Takes XML Schema and prints out internal XQuery type

Auxiliary research tools:

galax-mapwsdl

Imports/exports Galax queries as WSDL Web Services

xquery2soap

galax-project

Takes XQuery query and figures out what fragments of documents are necessary to evaluate the query