PrepServer Architecture

From RHESSI Wiki

Introduction

This is an overview of the complete PrepServer system. The client communicates with the front-end (PrepServer Front-End), which is connected to the middle-tier (PrepServer Management Service). This Management Service distributes the client pre-processing requests among the available PrepServer IDL Pre-Processors (back-end).

The PrepServer is a Java server application that allows for remote pre-processing of solar data. It currently pre-processes the following instruments:

• RHESSI
• SOHO/EIT
• STEREO/EUVI
• HINODE/XRT
• HINOTDE/EIS
• TRACE

The PrepServer is a three tier server application:

• The front-end service is a Java web application that runs in Tomcat.
• The middle-tier is a management service that bridges between the front-end and the back-end. It distributes pre-processing requests among the IDL pre-processing servers (back-end).
• The back-end servers are the actual pre-processing servers (currently only SSW IDL). These servers run independently and maintain their own IDL processes (idl_opserver). Thus, the PrepServer architecture allows for distributed and concurrent pre-processing.

Web Front-End

The Web Front-End is a Java web application running in a Tomcat. The service specification is available on Hesperia and Wilco in form of a WSDL file.

The Web Front-End has two main responsibilities:

• It is the single point of entry for clients using the pre-processing service. The Web Front-End's main method is preprocess, which takes a configured request package called PrepRequest. The PrepServer Web Front-End is responsible to keep the connection to the client open, while it internally sends the PrepRequest to the PrepServer Management Service.
• After a pre-processing request has been successfully completed, the Web Front-End publishes the level-1 data file in a web-accessible directory from where the data can be downloaded using regular utilities such as a browser, sock_copy, etc.

Management Service

This schema shows the Management Service architecture and how pre-processing requests are handled by it (distribution among idle Pre-Processing Services).

The Management Service is the intermediary between the Web Front-End and the Pre-Processing Services and is a separate service independent of the other components. It's main responsibilities are:

• Managing the Pre-processing Server connections by offering register and un-register routines.
• Keeping track of busy / idle Pre-processing Servers.
• Managing user pre-processing requests by distributing the requests among all idle Pre-processing Servers.
• Taking pre-processing requests with multiple data files attached, and splitting them into several one-data-file requests that can be distributed individually among idle pre-processing server.
• Providing a central service for the creation of Java PrepServer objects (POJO service)
• Providing a central service that handles uploads and downloads (Media service). This media service provides a centralized entry point for (all the data, regardless of their processing status, are sent to the public folder on the web server):
  • un-processed data upload and processed data download requests from users
  • un-processed data download requests from pre-processing servers and
  • processed data upload requests from pre-processing servers.

Pre-processing Service

Pre-processing Servers are SSW IDL enabled services that perform the actual pre-processing in SSW IDL.

Pre-processing usually involves the following steps:

1. Reading the pre-processing requests that are modeled in Java and converting them to an IDL readable set of parameters.
2. Downloading the un-processed data using the Media Service.
3. Sending the data file and all parameters to SSW IDL and invoking the appropriate pre-processing routines.
4. After the successful completion, uploading the now pre-processed data to the Web Front-End (again by using the Media Service).
5. Freeing the server and make it available to new requests.

Pre-processing Servers all maintain their own IDL session (idl_opserver) and are therefore independent from one another. This allows for multiple pre-processing servers on the same machine, each occupying another CPU. Should more processing power be needed, pre-processing servers can be started on distributed machines and configured to register with an already available centralized Management Service.

Pre-processing Servers register at startup with the Management Service and un-register upon receiving a termination signal. They do AFTER completing their current pre-processing requests.