OpenPAP

Introduction

OpenPAP is a system that produces graphical complex weather elements for newspapers. The majority of the system may run on many platforms but this release is focused on Windows (XP) only.

Requirements

Windows XP

Java JRE 1.6 or higher

PostgreSQL 8.3 or higher

Microsoft .NET Framework 2.0 or higher

Adbobe InDesign CS3 or higher

Components

PostgreSQL database dump, weather database

OpenPAPXml.jar - command line program to produce XML for Abode InDesign

OpenPAPProducer - Windows program, runs the automated production against Adobe InDesign

OpenPAPCorrGUI.jar - Java GUI program, edit the productiontable in the database

OpenPAPAdmGUI.jar - Java GUI program, make new customers, and select what data that should be available to them

Datasources

NAAF - pollen data

api.met.no - forecast data, uv data, extreme observation data

eKlima - observation data

weather db - facttext data

Sjøkartverket/weather db - Tidal water

Download

http://kundedb.dnmi.no/OpenPAP.zip

Installation

The requirements

Java JRE 1.6:

See http://java.sun.com

Adobe InDesign CS3:

See http://www.adobe.com

.Net framework 2.0:

See http://windowsupdate.microsoft.com

PostgreSQL 8.3:

Download PostgreSQL from http://pginstaller.projects.postgresql.org and follow their installation guide.

You don't need “National language support”, “PostGIS”, “PL/Java” or “psql”, but you do need “Data directory” and “pgadmin III”. If you're unsure, just install everything.

Install it as a service

Accept connections on all addresses, not just on localhost - If you are planning to run the system on not just one computer

Locale: C Encoding: LATIN-5

superuser: postgres password: Post#Gres!1 (this is the default username/password in the system configuration) - should/may be changed

No contrib modules is needed

The OpenPAP package

Extract OpenPAP.zip into c:\. This will result in the following structure:

c:\OpenPAP\
c:\OpenPAP\distfromhere
c:\OpenPAP\prodxml
c:\OpenPAP\bat\
c:\OpenPAP\bat\postbat__general.bat
c:\OpenPAP\bat\prebat__general.bat
c:\OpenPAP\bat\queue_Test_Oslo.bat
c:\OpenPAP\bin\
c:\OpenPAP\bin\OpenPAPXml.jar
c:\OpenPAP\bin\OpenPAPCorrGUI.jar
c:\OpenPAP\bin\OpenPAPAdmGUI.jar
c:\OpenPAP\bin\OpenPAP-Tidalwater_parser.pl
c:\OpenPAP\config\
c:\OpenPAP\config\OpenPAP.properties
c:\OpenPAP\customers\
c:\OpenPAP\customers\Test_Oslo\
c:\OpenPAP\customers\Test_Oslo\Test_Oslo.ai
c:\OpenPAP\customers\Test_Oslo\Test_Oslo.indd
c:\OpenPAP\customers\Test_Oslo\graphics\
c:\OpenPAP\customers\Test_Oslo\graphics\<lots of symbol files>
c:\OpenPAP\dbdump\
c:\OpenPAP\dbdump\weather.backup
c:\OpenPAP\install\
c:\OpenPAP\install\setup.exe
c:\OpenPAP\install\Setup.msi
c:\OpenPAP\log\
c:\OpenPAP\output\
c:\OpenPAP\queue\
c:\OpenPAP\queue_specialprod\
c:\OpenPAP\tmpQueue\
c:\OpenPAP\xml\

Database:

Restore the default weather database into your new PostgreSQl 8.3

Start pgadmin III from the Windows start menu

The localhost server is normally already added. Just connect to it and type in the password (defaul Post#Gres!1). But if it's not added do this: Go to menu “File” - > “Add server” and then type in the database server properties (default - Name: localhost, Host: localhost, Port: 5432, Maintenance DB: postgres, Username: popstgres, Password: Post#Gres!1). Also click “Store password” and connect now before “OK” button.

Select “Databases” belonging to localhost in the Object browser window. Go to menu “Edit” -> “New Object” -> “New Database”. In the following window, type in the name “weather” in the name textbox and click OK (nothing more!).

Select the new database “weather” in the Object browser by clicking it once. Go to menu “Tools” - > “Restore. Then browse and select c:\OpenPAP\dbdump\weather.backup. Only “Verbose messages” should be selected. Click OK, and then “Done” when it appears.

Exit pgadmin III

Config:

Open c:\OpenPAP\config\OpenPAP.properties and edit some of the fields

Edit smtp server by changing the value of after smtp. For instance: smtp=smtp.chello.no

Edit OpenPAPXml_email_to (multiple as commaseparated) and OpenPAPXml_email_from. This is email warning when something goes wrong during XML production.

Edit database properties if you have installed the database on another machine or you have not been using the usernam/password as suggested in the install procedure.

Edit OpenPAPProducer_email_to (multiple as commaseparated) and OpenPAPProducer_email_from. This is email warning when something goes wrong during InDesign production.

Edit the path's if you have different file structure

OpenPAPProducer:

Start c:\OpenPAP\install\setup.exe and follow instructions

Usage

Example customer

The example customer Test_Oslo with customerid 1 comes with the package. It's a good idea to test your installation by producing this customer. This is how you proceed:

Run OpenPAPProducer

Select customer “Test_Oslo” from the dropdown menu and clik the appearing “Produce Test_Oslo” button.

The OpenPAPProducer window provides status and progressinfo. First you see a DOS window, this is OpenPAPXml prepearing the XML file with data for Adbobe Indesign. Then Abobe InDesign starts and run's the template for Test_Oslo. It finnishes by exporting PDF and EPS into the output directory. If not changed this is c:\OpenPAP\output\Test_Oslo.pdf and c:\OpenPAP\output\Test_Oslo.eps.

Make your own

If the example runned smoothly you are ready to produce your very own product.

Find out what data you would like to include in the final product

Create a static background picture in for instance Adobe Illustrator. This would normally contain

empty maps

tables containing nothing but location names and columnnames

empty space for facttexts, textforecasts and so on

Depending on what data you want, you may need some extra graphics, for instance weather symbols and wind arrows. All kinds of extra graphics are located in the graphics folder for the test-customer. Take a look there to understand the file naming and create your own sets.

Start OpenPAPAdmGUI

Create the customer

Add metadata for the customer, for instance model data for a selected amount of locations, parameters and days. Metagroup is best used by using names based on each graphical element in the Adobe InDesign template. For instance, if you are selecting stations and parameters for a Europe weather map, set metagroup to “europa_map”. This will make it easier to find the data in the XML.

Add stations inn the station if you can't find your desired station/location.

Create a customer folder in c:\OpenPAP\customers\ - The folder name have to be the same as the customer name in chosen in OpenPAPAdmGUI.

Now you are ready to produce the XML for Adbobe InDesign

Start OpenPAPProducer

Select your customer from the dropdown menu and clik the appearing Produce button. This will fail after creating XML because Adobe Indesign is not set up yet. This is OK. We still get the XML and that's all we want now. The XML should be located in c:\OpenPAP\customers\<customername>. Leave it for now.

Start Abobe InDesign.

Create a new blank template.

Add the background image created earlier, as a layer

Provide boxes for every spot you wish to fill with data. The boxes are made by using “Rectangle frame tool”. You can scale all boxes in a layer so make sure you create a layer for each row in a table, or at least a layer for each table. For instance, all weather symbols should be in one separate layer.

Import XML from File menu. The XML is located in c:\OpenPAP\customers\<customername>

Start dragging from XML and dropping on each element in your Adobe InDesign template.

You may experience that the weather symbols are too large, this can be fixed by naming the layer they belong to with a reserved ending, ” scale [1-100]“. If you want the symbols on layer “symbol long term” to be 40% of their size; rename the layer to “symbol long term scale 40”. The symbols will then adjust automaticly when beeing produced.

If you wish to turn som layers on or off at a spesific time in the year you may do this in the OpenPAPAdmGUI program. Write down the layer names you wish to turn on/off.

Save template and exit Adobe InDesign.

Start OpenPAPAdmGUI

Go to the “Indesign layer” pane

Select you customer and click “New”

Write the layer name and make sure that active is selected. That makes this layer active. Type in start- and enddate in the ddMM format, for instance 2901 for January 29. You can also choose wintertime or summertime as start- and enddate.

Start OpenPAPProducer

Select your customer from the dropdown menu and clik the appearing Produce button.

The OpenPAPProducer window provides status and progressinfo. First you see a DOS window, this is OpenPAPXml prepearing the XML file with data for Adbobe Indesign. Then Abobe InDesign starts and run's the template for your customer. It finnishes by exporting PDF and EPS into the output directory, c:\OpenPAP\output\

Production

Scheduled production

When you are all set to produce the customer you may want to do it automated at specific times. This can be done by using “Microsoft Windows scheduled tasks”. OpenPAPProducer looks in c:\OpenPAP\queue\ every ten seconds for a file. If the file name is numeric it will start producing the customer with the matching customer id as the file name. This is how you do it:

Make a bat-file that writes an empty file with customer id as file name to c:\OpenPAP\queue\. This is done by copying the existing file c:\OpenPAP\bat\queue_Test_Oslo.bat to c:\OpenPAP\bat\queue_<customername>.bat.

Then edit your new c:\OpenPAP\bat\queue_<customername>.bat and replace the number 1 with your customer id

Open “scheduled tasks” and add a new scheduled task. The file to run is c:\OpenPAP\bat\queue_<customername>.bat. Set the times, for instance 16:00 each day.

Start OpenPAPProducer

Test the production by manually running c:\OpenPAP\bat\queue_<customername>.bat or wait for the scheduled task to run it. Then OpenPAPProducer will start the production and you will find the outputfiles in c:\OpenPAP\output\ after production is completed.

Manual production

Start OpenPAPProducer

Select customer and click the produce button

Special production - OpenPAPCorrGUI Produce

This option shall be used after doing corrections in the customer data with OpenPAPCorrGUI. OpenPAPCorrGUI is typically used after a production with reported some missing data. Use OpenPAPCorrGUI to edit the missing/incorrect data.

Start OpenPAPCorrGUI

Select the customer that need data edit

Edit the missing/incorrect data and save

Start OpenPAPProducer

If you start a normal production for this customer the data you corrected will be overwritten with missing/incorrect data. Select customer and then, instead of the normal Produce button, press the smaller “Produce with manually corrected data”-button below.

Data and parameters supported

(1) Model data, forecast

Parameter id	Parameter Name	Datestart	Dateend
8206	GRAPHICS_INDT	0	9
8445	TEXTFORECAST	0	8
8900	GRILLINDEX	0	9
9885	WINDARROW_8_COLOR	0	9
9985	WINDARROW_8	0	9
9987	WIND_INTERVAL	0	9
9988	PREC_INTERVAL	0	9
9990	WEATHERTXT	0	9
9991	WINDTYPE_TXT	0	9
9994	PRECIPITATION	0	9
9995	WIND_SPEED	0	9
9996	TEMPERATURE	0	9
9998	WINDARROW	0	9
9999	WSYMBOL	0	9

(5) Sun Moon Tidalwater

Parameter id	Parameter Name	Datestart	Dateend
8000	TV_H1	-n	n
8001	TV_H2	-n	n
8002	TV_L1	-n	n
8003	TV_L2	-n	n
8004	TV_H1_CM	-n	n
8005	TV_H2_CM	-n	n
8006	TV_L1_CM	-n	n
8007	TV_L2_CM	-n	n
8101	SUN_UP	-n	n
8102	SUN_DOWN	-n	n
8103	MOON_UP	-n	n
8104	MOON_DOWN	-n	n
8106	SUN_LENGTH	-n	n
8107	DAYS_SUN_RETURN	1	1
8108	SINCE_S_W_PHASEDESC	1	1

(6) Observations

Parameter id	Parameter Name	Datestart	Dateend
9989	SNOW_DEPTH	-n	0
9990	WEATHERTXT	-n	0
9991	WINDTYPE_TXT	-n	0
9992	TEMPERATURE_MIN	-n	0
9993	TEMPERATURE_MAX	-n	0
9994	PRECIPITATION	-n	0
9995	WIND_SPEED	-n	0
9996	TEMPERATURE	-n	0
9998	WINDARROW	-n	0
9999	WSYMBOL	-n	0
9820	MULTIRES_WARMEST	0	0
9821	MULTIRES_COLDEST	0	0
9822	MULTIRES_MOST_WET	0	0
9823	MULTIRES_MOST_WINDY	0	0
9824	MULTIRES_MOST_SUNNY	0	0

(7) Moon phases

Parameter id	Parameter Name	Datestart	Dateend
8105	GRAPHICS_MOONFACE	-n	n

(8) Fact texts

Parameter id	Parameter Name	Datestart	Dateend
8400	FACTTEXT	1	1

(14) Time - days, dates

Parameter id	Parameter Name	Datestart	Dateend
9000	DAYOFWEEK_FULL	-n	n
9001	DAYOFWEEK_3CHAR	-n	n
9002	DAYOFWEEK_2CHAR	-n	n
9010	MONTH_FULL	-n	n
9011	MONTH_3CHAR	-n	n
9020	DATES_YEAR	-n	n
9021	DATES_YEAR_SHORT	-n	n
9022	DATES_MONTH	-n	n
9023	DATES_DAY	-n	n
9024	DATES_FULL	-n	n

(16) Pollen forecast

Parameter id	Parameter Name	Datestart	Dateend
8109	POLLEN_OR	0	1
8110	POLLEN_HASSEL	0	1
8111	POLLEN_BJØRK	0	1
8112	POLLEN_GRESS	0	1
8113	POLLEN_BUROT	0	1

(20) UV forecast

Parameter id	Parameter Name	Datestart	Dateend
8210	GRAPHICS_UVINDEX	0	3

Overview

Administration

Production

Features

Tidalwater - Special handling

Tidalwater is not a free source and is not available with HTTP GET or web services, so you have to import an ordered file into the database with a special script. This is for norwegian tidalwater data:

Contact “Sjøkartverket” and get a file for each position you want. The files contain time and cm for each low and high tide, for each day in a year.

Run the tidalwater parser from commandline. perl needs to be installed to run it. Run command: “c:\OpenPAP\bin\OpenPAP-tidalwater_parser.pl <inputfile> <stationid>”

This should result in a series of INSERT statements. Copy this into PgAdmin III and run them in a SQL window.

Tidalwater is now in the database and you are free to use tidalwater data.

Pollen data - Special handling

To get the norwegian pollen data from NAAF into pap you will have to contact NAAF for a user id.

Place the userId into c:\OpenPAP\config\OpenPAP.properties like this: naaf_userid = userid

TODO - made 2008-04-21

making distributable symb/etc ai-files for example

PapScriptHandler, .Net 2.0 implementation

Database connection OpenPAPProducer

Tidalwater to XML

Design adminGUI

Pollen forecast

OpenPAPProducer - produce with manually corrected data

Tidalwater parse to database script

Making example customer

Gereral config file for all programs in package.

Pollen auth

Modeldata auth

Fikse inkonsistens mellom yr og eklima for extreemes

Textforecast

Testserver into DMZ

Implementation adminGUI

Documentation

Test: install and testrun of example customer

Test: setup of new customer and scheduled run

Javadoc in OpenPapXml.jar

Open source info in code and wiki page

SVN import

Last fixes in OpenPAPCorrAPP.jar

Last fixes in OpenPAPAdmAPP.jar

Last fixes in OpenPAPXml.jar

Last fixes in weather database

Last fixes in OpenPAPProducer

Datavalue check, quality-assured

General testing

Bugfix