Amazon Web Service as a Datasource
Updated September 23, 2009
Using Web Services as a Jaspersoft Data Source
An Example using Amazon E-Commerce Service (ECS)
Disclaimer
This document and the code that accompany it is provided AS-IS.
The information contained in this document is the proprietary and exclusive
property of Jaspersoft except as otherwise indicated. No part of this document, in
whole or in part, may be reproduced, stored, transmitted, or used for design
purposes without the prior written permission of Jaspersoft.
The information contained in this document is subject to change without notice.
Jaspersoft specifically disclaims all warranties, express or limited, including, but
not limited, to the implied warranties of merchantability and fitness for a particular
purpose, except as provided for in a separate software license agreement.
Privacy Information
This document may contain information of a sensitive nature. This information
should not be given to persons other than those who are involved in the training
or who will become involved during the development of reports.
Author: Luke Shannon
Profession Services Consultant, Jaspersoft April 16, 2007
Updated: Mary Flynn
Sales Engineer, Jaspersoft September 23, 2009
Amazon Web Service as a Datasource
Updated September 23, 2009
Table of Contents
Using Web Services as a Jaspersoft Data Source...1
An Example using Amazon E-Commerce Service (ECS)...1
Disclaimer...1
Privacy Information...1
Table of Contents...2
Introduction...2
Intended Audience...2
Required Resources...3
iReport...3
Copy the required JAR libraries...3
Register the Query Executer...4
Set the Datasource...5
Run the Report...6
Deploying in JasperServer...8
Copy the required JAR libraries...8
Register the Query Executer...8
Deploy the Report Unit...9
Run the Report...14
Deploying in a Standard Web Application...15
Sample Code...17
AmazonQueryExecuterFactory.java...17
AmazonWebServicesQueryExecuter.java...18
AmazonWebServiceCall.java...20
SignedRequestsHelper.java...23
Introduction
This document uses Amazon E-Commerce Services (ECS) as an example of using web
services as a Jaspersoft data source. This sample code:
Uses a custom Query Executer to obtain the data from a web service call (a
URL).
Converts the XML document returned by the web service call into a data source
that Jaspersoft can consume.
iReport, JasperServer, or your own application fills and renders the finished report. This
document describes how to deploy the sample so it can be used by iReport,
JasperServer, and in stand-alone web applications.
Intended Audience
To understand the examples in this document the reader should have a working
knowledge of the JasperReports API and Java development for the web.
Required Resources
In order to run this sample, the following jars must be in your application’s classpath. (xx
is a placeholder for the version number.)
AmazonQueryExecuter.jar
o
Custom JasperReports data source and query executer
o
Source included with this sample
commons-codec-xx.jar
o
Implementations of common encoders and decoders
o
http://commons.apache.org/codec
o
Note: Amazon requires signed web services requests
jdom-xx.jar
o
Java-oriented object model which models XML documents
o
http://www.jdom.org
jasperreports-xx.jar and its dependent files
o
http://www.jasperforge.org
An Amazon Web Services account (free)
o
http://aws.amazon.com/
Amazon Account
Create an account
If you have purchased anything from Amazon in the past, you may already have an
Amazon account. In this case you may simply activate web services. If you are new to
Amazon, the account may be created in a few minutes.
Login to your account
Login at http://aws.amazon.com/ and click Your Account -> Security Credentials
Note your Access Key ID and Secret Access Key. You will need these to run the sample
reports.
iReport
Copy the required JAR libraries
2. Copy the following files from the sample
\libs
directory to the same location as in
step #1 above.
commons-codec-xx.jar
jdom-xx.jar
3. Start iReport. Click Tools > Options.
Note: The first screen provides iReport settings. The General … Miscellaneous
screens are for global NetBeans settings.
4. Click the Classpath tab and click Add JAR. Navigate to
<ireport-install-dir>\ireport\libs
and add each of the following. Click the Reloadable
checkboxes and click OK when done.
commons-codec-xx.jar
jdom-xx.jar
AmazonQueryExecuter.jar
Register the Query Executer
2. Click the Query Executers tab and click Add.
3. Enter the following and click OK.
Language
amazonECS
Factory
class
com.jaspersoft.ps.amazondemo.AmazonQueryExecuterFactory
Notes:
You can specify anything you like for the language, but the factory class needs to
match the name of the Factory packaged in AmazonQueryExecuter.jar.
The sample report uses the language "amazonECS", so you must use this name
if you want the unmodified sample report to work correctly.
Set the Datasource
2. Select Query Executer Mode and click Next.
3. Enter a name for the query executer, for example: amazonECS_test.
Note: The name you give the Query Executer datasource does not matter. The
report includes the name of the Query Language (“amazonECS”) and you already
registered the amazonECS Query Language with the Query Executer factory.
Run the Report
1. Open the sample file included with this distribution.
3. The query language should be set to “amazonECS” to match the query language
definition.
4. Notice the query syntax looks like XPath and is treated as such when the report
is run against the custom Amazon ECS Query Executer. Click Cancel to exit this
screen.
Deploying in JasperServer
Having this report run on JasperServer is simply obtained in just a few steps.
Copy the required JAR libraries
1. Shutdown JasperServer.
2. Copy AmazonQueryExecuter.jar from the sample
\dist
directory to
<jasperserver-install-dir>\
jasperserver-pro\WEB-INF\lib
.
Register the Query Executer
Registering a query executer in JasperServer is done directly in a text file located on
your web server. You need write access to privileges to the server to accomplish this
step.
1. Use a text editor or your IDE (e.g., Eclipse, NetBeans) to modify the
jasperreports.properties
file. This file is located:
2. Add the following lines and save. Make sure you’re saving in plain text format.
# Sample query executer that retrieves data using web services
net.sf.jasperreports.query.executer.factory.amazonECS=com.jaspersoft.ps.amazon demo.AmazonQueryExecuterFactory
3. Start JasperServer.
Deploy the Report Unit
These instructions show how to deploy the Report Unit using the JasperServer web
interface. (You can also deploy the Report Unit in iReport using the Repository
Navigator pod.)
1. Launch a web browser and navigate to your JasperServer instance, for example:
http://localhost:8080/jasperserver-pro.
2. Login with administrator or superuser rights (the JasperServer login page
displays the default login credentials for jasperadmin and superuser).
3. Click View > Repository. Navigate to a folder in which you’ll store the report unit.
In the screen shots below, the report was saved to Public so all users can access
it.
5. Enter the following and click Next:
Name
AmazonBooksByAuthor1
Label
Amazon Books by Author 1
6. Upload the sample JRXML supplied with this document.
7. Click None for the Data Source and click Next.
Note: the data source is registered with the query language defined in the report.
8. Select none for the Query and click Next.
9. Click Next to skip the Custom Report View. Click Finish, then Save.
Shortcut: you may click "Finish" on step 6 and skip the final steps of the wizard
because we don't need to set anything in the final steps.
Create and link the input controls
Input controls may be added directly to a report or created separately and then linked to
a report. As a best practice it is better to create the input control separately if it will be
needed in more than one report.
We will create three input controls:
AuthorName
1. Add the new input control
3. Edit the report and link the input controls to it.
Repeat for AWSAccessKey and AWSSecretKey.
Run the Report
Run Report 2
Load the second report to JasperServer just like the first one. The only difference is that
this report makes use of an image file. So you will need to first upload the image
Deploying in a Standard Web Application
You can use the following sample code in your own web application provided that all
required resources are in place and a properties file is in the WEB-INF/classes folder of
the application. It is up to the developer implementing this to determine how to obtain
the author name, etc.
File reportFile = new File(context.getReal-Path("/jrxml/simpleReport2.jasper"));
Map<String, Object> parameters = new HashMap<String, Object>();
//put in the directory
parameters.put("XML_AUTHOR_NAME",authorName);
parameters.put("IS_IGNORE_PAGINATION", Boolean.FALSE ); if (!reportFile.exists()) {
throw new JRRuntimeException("File student_report.-jasper not found. The report design must be compiled first.");
}
//create a print object
JasperReport jr = (JasperReport)JRLoader.loadObject(report-File.getPath());
//create a print object
Sample Code
AmazonQueryExecuterFactory.java
package com.jaspersoft.ps.amazondemo; import java.util.Map; import net.sf.jasperreports.engine.JRDataset; import net.sf.jasperreports.engine.JRException; import net.sf.jasperreports.engine.query.JRQueryExecuter; import net.sf.jasperreports.engine.query.JRQueryExecuterFactory; /*** Executer factory. See JasperReports Ultimate Guide,
* Section Reporting Data : Report Queries : Query Executer API. *
* @author lshannon */
public class AmazonQueryExecuterFactory implements JRQueryExecuterFactory { /**
* Not doing anything with this. * From JasperReports Ultimate Guide:
* Returns parameters that will be automatically registered with a * report/dataset based on the query language. These parameters are * used by query executers as the context/connection on which to * execute the query. For instance, the Hibernate query executer * factory specifies a HIBERNATE_SESSION parameter of type
* org.hibernate.Session whose value will be used by the query executer * to run the query.
* @return */
public Object[] getBuiltinParameters() { return new Object[] {};
}
/**
* Creates a query executer. The dataset includes the query string * and the fields that will be requested from the data source created * by the query executer. The parameters map contains parameter
* types and runtime values to be used for query parameters. This method * usually sends the dataset and parameters map to the created query executer.
* @param jRDataset * @param map
* @return
* @throws JRException */
public JRQueryExecuter createQueryExecuter(JRDataset jRDataset, Map map) throws JRException {
return new AmazonWebServicesQueryExecuter(jRDataset, map); }
/**
* From JasperReports Ultimate Guide: Used on report validation to
* determine whether a query parameter type (for a parameter specified in * the query using $P{..}) is supported by the query executer
implementation. * @param arg0 * @return */
public boolean supportsQueryParameterType(String arg0) { return true; } }
AmazonWebServicesQueryExecuter.java
package com.jaspersoft.ps.amazondemo; import java.util.Map; import net.sf.jasperreports.engine.JRDataSource; import net.sf.jasperreports.engine.JRDataset; import net.sf.jasperreports.engine.JRException; import net.sf.jasperreports.engine.data.JRXmlDataSource; import net.sf.jasperreports.engine.query.JRAbstractQueryExecuter; import org.w3c.dom.Document; /*** Responsible for running a query and creating a data source from the result. * Works with REST. Some work may be needed to get it to work with SOAP.
*
* @author lshannon */
public class AmazonWebServicesQueryExecuter extends JRAbstractQueryExecuter {
/**
* Get a reference to the parent class.
* Abstract base provides query parameter processing functionality * and other utility methods.
* @param jrdataset * @param parameterMap */
protected AmazonWebServicesQueryExecuter(JRDataset jrdataset, Map parameterMap) {
super(jrdataset, parameterMap); parseQuery();
}
/**
* Processes and runs the query and creates a data source out of the query
* @throws JRException */
public JRDataSource createDatasource() throws JRException { JRXmlDataSource datasource = null;
String xPath = getQueryString();
// TODO for production: use logging rather than System.out.println System.out.println("Query string: " + xPath);
System.out.println("Executing using author: " + (String)getParameterValue("XML_AUTHOR_NAME"));
Document document =
AmazonWebServiceCall.execute((String)getParameterValue("XML_AUTHOR_NAME")); if (document != null && xPath != null)
{
datasource = new JRXmlDataSource(document, xPath); }
return datasource; }
/**
* Not implementing this.
* Closes the query execution results and any other resource associated * with it. This method is called after all data produced by the query * executer has been fetched.
*/
public void close() { }
/**
* Not implementing this.
* Called when the user decides to cancel a report fill process. The * implementation should check whether the query is currently being * executed and ask the underlying mechanism to abort the execution. The * method should return true if the query was being executed and the * execution was canceled. If execution cancellation is not supported, * the method will always return false.
* @return
* @throws JRException */
public boolean cancelQuery() throws JRException { return true;
}
/**
* Not implementing this. * @param arg0
* @return */
@Override
protected String getParameterReplacement(String arg0) { return null;
}
}
package com.jaspersoft.ps.amazondemo; import java.io.FileWriter; import java.io.IOException; import java.io.InputStream; import java.io.InputStreamReader; import java.io.PrintStream; import java.io.PrintWriter; import java.util.HashMap; import java.util.Map; import net.sf.jasperreports.engine.JRException; import net.sf.jasperreports.engine.util.JRXmlUtils; import org.jdom.JDOMException; import org.jdom.input.SAXBuilder; import org.jdom.output.DOMOutputter; import org.jdom.output.Format; import org.jdom.output.XMLOutputter; /**
* Receives Author Name from the user, combines it with
* hard-coded sample parameters, calls SignedRequestHelper to construct * the URL, runs the web service call, and returns a w3c dom document. * The dom document is an XML file, which JasperReports can consume with * the XML data source.
* Updated 2009-09-27 by mflynn: replaced ECSHelper with SignedRequestHelper. * @author lshannon
*/
public class AmazonWebServiceCall {
/**
* All other inputs are hard-coded for simplicity.
* For production, take a Map of properties to use in the call to the service.
* @author lshannon
* @param The name of the author to perform the ItemSearch on. */
private static final String AWS_ACCESS_KEY_ID = "YOUR_ACCESS_ID_GOES_HERE";
private static final String AWS_SECRET_KEY = "YOUR_AWS_SECRET_KEY_GOES_HERE";
private static final String ENDPOINT = "ecs.amazonaws.com"; private static final String ITEMPAGE = "1";
private static final String OPERATION = "ItemSearch";
private static final String RESPONSEGROUP = "SalesRank,Small"; private static final String SEARCHINDEX = "Books";
private static final String SERVICE = "AWSECommeceService"; private static final String SORT = "salesrank";
private static final String VERSION = "2009-03-31";
/**
* @param author * @return
*/
@SuppressWarnings("deprecation")
public static org.w3c.dom.Document execute(String author) { /*
* Set up the signed requests helper. */
SignedRequestsHelper helper; try {
helper = SignedRequestsHelper.getInstance
(ENDPOINT, AWS_ACCESS_KEY_ID, AWS_SECRET_KEY); } catch (Exception e) {
e.printStackTrace(); return null;
}
String requestUrl = null;
/*
* Here the request parameters are stored in a map. */
System.out.println("Map form example:");
Map<String, String> params = new HashMap<String, String>(); params.put("Author", author); params.put("ItemPage", ITEMPAGE); params.put("Operation", OPERATION); params.put("ResponseGroup", RESPONSEGROUP); params.put("SearchIndex", SEARCHINDEX); params.put("Service", SERVICE); params.put("Sort", SORT); params.put("Version", VERSION);
requestUrl = helper.sign(params);
System.out.println("Signed Request is \"" + requestUrl + "\"");
org.w3c.dom.Document doc; try {
org.jdom.Document jdom = toJDOM(invokeURL(requestUrl)); //write it out to ensure we are getting something
writeOutPrettyFile(jdom);
System.out.println("Document written to file system"); doc = JRXmlUtils.parse(invokeURL(requestUrl));
} catch (Exception e) { doc = null;
System.out.println("Opps: " + e); }
System.out.println("Returning the document"); return doc;
}
/**
* the local file system. Only works with the JDom. *
* @param doc
* @throws IOException */
private static void writeOutPrettyFile(org.jdom.Document doc) throws IOException {
FileWriter fw;
fw = new FileWriter(
"C:\\amazon.xml", false);
PrintWriter pw = new PrintWriter(fw, true); Format format = Format.getPrettyFormat(); XMLOutputter out = new XMLOutputter(format); out.output(doc, pw);
}
/**
* This makes the call to the Web Service url *
* @param url * @return */
public static InputStream invokeURL(String url) { java.net.URL u;
java.net.URLConnection conn; try {
u = new java.net.URL(url); conn = u.openConnection(); return conn.getInputStream(); } catch (Exception e) {
e.printStackTrace(); }
System.out
.println("There was an issue hitting the web service. Returning null.");
return null;
}
/**
* Writes out the input stream. No processing or formatting. *
* @param in * @param out
* @throws java.io.IOException */
public static void dumpResponse(InputStream in, PrintStream out) throws java.io.IOException {
InputStreamReader r = new InputStreamReader(in); int ch;
while ((ch = r.read()) != -1) { out.write((char) ch); }
/**
* Convert the input stream into a JDOM document. *
* @param in * @return
* @throws JDOMException * @throws IOException */
public static org.jdom.Document toJDOM(InputStream in) throws JDOMException, IOException {
SAXBuilder builder = new SAXBuilder(); return builder.build(in);
}
/**
* Convert the input stream into a DOM document. *
* @param in * @return
* @throws JRException */
public static org.w3c.dom.Document toDOM(java.io.InputStream in) throws JRException {
return JRXmlUtils.parse(in); }
/**
* Convert a JDom to a DOM document. *
* @param doc * @return
* @throws JDOMException */
public static org.w3c.dom.Document toDOM(org.jdom.Document doc) throws JDOMException {
DOMOutputter outputter = new DOMOutputter();
org.w3c.dom.Document document = outputter.output(doc); return document;
}
/** *
* @param args */
public static void main(String args[]) {
AmazonWebServiceCall.execute("William Gibson"); }
SignedRequestsHelper.java
/
****************************************************************************** ****************
* Copyright 2009 Amazon.com, Inc. or its affiliates. All Rights Reserved. *
* Licensed under the Apache License, Version 2.0 (the "License"). You may not use this file
* except in compliance with the License. A copy of the License is located at *
* http://aws.amazon.com/apache2.0/ *
* or in the "LICENSE.txt" file accompanying this file. This file is distributed on an "AS IS"
* BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations under the License.
* *
****************************************************************************** **************
*
* Amazon Product Advertising API * Signed Requests Sample Code *
* API Version: 2009-03-31 *
*/
package com.jaspersoft.ps.amazondemo;
// package com.amazon.advertising.api.sample;
import org.apache.commons.codec.binary.Base64;
/**
* Contains all the logic for signing requests * to the Amazon API.
*/
public class SignedRequestsHelper { /**
* All strings are handled as UTF-8 */
private static final String UTF8_CHARSET = "UTF-8";
/**
* The HMAC algorithm required by Amazon */
private static final String HMAC_SHA256_ALGORITHM = "HmacSHA256";
/**
* This is the URI for the service, don't change unless you really know * what you're doing.
*/
private static final String REQUEST_URI = "/onca/xml";
/**
* The sample uses HTTP GET to fetch the response. If you changed the sample
* to use HTTP POST instead, change the value below to POST. */
private static final String REQUEST_METHOD = "GET";
private String endpoint = null;
private String awsAccessKeyId = null; private String awsSecretKey = null;
private SecretKeySpec secretKeySpec = null; private Mac mac = null;
/**
* You must provide the three values below to initialize the helper. *
* @param endpoint Destination for the requests. * @param awsAccessKeyId Your AWS Access Key ID
* @param awsSecretKey Your AWS Secret Key * @return
* @throws IllegalArgumentException * @throws UnsupportedEncodingException * @throws NoSuchAlgorithmException * @throws InvalidKeyException */
public static SignedRequestsHelper getInstance( String endpoint,
String awsAccessKeyId, String awsSecretKey
) throws IllegalArgumentException, UnsupportedEncodingException, NoSuchAlgorithmException, InvalidKeyException
if (null == endpoint || endpoint.length() == 0)
{ throw new IllegalArgumentException("endpoint is null or empty"); }
if (null == awsAccessKeyId || awsAccessKeyId.length() == 0)
{ throw new IllegalArgumentException("awsAccessKeyId is null or empty"); }
if (null == awsSecretKey || awsSecretKey.length() == 0)
{ throw new IllegalArgumentException("awsSecretKey is null or empty"); }
SignedRequestsHelper instance = new SignedRequestsHelper(); instance.endpoint = endpoint.toLowerCase();
instance.awsAccessKeyId = awsAccessKeyId; instance.awsSecretKey = awsSecretKey;
byte[] secretyKeyBytes = instance.awsSecretKey.getBytes(UTF8_CHARSET); instance.secretKeySpec = new SecretKeySpec(secretyKeyBytes,
HMAC_SHA256_ALGORITHM);
instance.mac = Mac.getInstance(HMAC_SHA256_ALGORITHM); instance.mac.init(instance.secretKeySpec);
return instance; }
/**
* The construct is private since we'd rather use getInstance() */
private SignedRequestsHelper() {}
/**
* This method signs requests in hashmap form. It returns a URL that should
* be used to fetch the response. The URL returned should not be modified in
* any way, doing so will invalidate the signature and Amazon will reject * the request.
* @param params * @return
*/
public String sign(Map<String, String> params) {
// Let's add the AWSAccessKeyId and Timestamp parameters to the request.
params.put("AWSAccessKeyId", this.awsAccessKeyId); params.put("Timestamp", this.timestamp());
// The parameters need to be processed in lexicographical order, so we'll
// use a TreeMap implementation for that.
SortedMap<String, String> sortedParamMap = new TreeMap<String, String>(params);
// get the canonical form the query string
String canonicalQS = this.canonicalize(sortedParamMap);
REQUEST_METHOD + "\n" + this.endpoint + "\n" + REQUEST_URI + "\n" + canonicalQS;
// get the signature
String hmac = this.hmac(toSign);
String sig = this.percentEncodeRfc3986(hmac);
// construct the URL String url =
"http://" + this.endpoint + REQUEST_URI + "?" + canonicalQS + "&Signature=" + sig;
return url; }
/**
* This method signs requests in query-string form. It returns a URL that * should be used to fetch the response. The URL returned should not be * modified in any way, doing so will invalidate the signature and Amazon * will reject the request.
* @param queryString * @return
*/
public String sign(String queryString) {
// let's break the query string into it's constituent name-value pairs Map<String, String> params = this.createParameterMap(queryString);
// then we can sign the request as before return this.sign(params);
}
/**
* Compute the HMAC. *
* @param stringToSign String to compute the HMAC over. * @return base64-encoded hmac value.
*/
private String hmac(String stringToSign) { String signature = null;
byte[] data; byte[] rawHmac; try {
data = stringToSign.getBytes(UTF8_CHARSET); rawHmac = mac.doFinal(data);
Base64 encoder = new Base64();
signature = new String(encoder.encode(rawHmac)); } catch (UnsupportedEncodingException e) {
throw new RuntimeException(UTF8_CHARSET + " is unsupported!", e); }
return signature; }
/**
*
* @return ISO-8601 format timestamp. */
private String timestamp() { String timestamp = null;
Calendar cal = Calendar.getInstance();
DateFormat dfm = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss'Z'"); dfm.setTimeZone(TimeZone.getTimeZone("GMT"));
timestamp = dfm.format(cal.getTime()); return timestamp;
}
/**
* Canonicalize the query string as required by Amazon. *
* @param sortedParamMap Parameter name-value pairs in lexicographical order.
* @return Canonical form of query string. */
private String canonicalize(SortedMap<String, String> sortedParamMap) { if (sortedParamMap.isEmpty()) {
return ""; }
StringBuffer buffer = new StringBuffer(); Iterator<Map.Entry<String, String>> iter = sortedParamMap.entrySet().iterator();
while (iter.hasNext()) {
Map.Entry<String, String> kvpair = iter.next();
buffer.append(percentEncodeRfc3986(kvpair.getKey())); buffer.append("=");
buffer.append(percentEncodeRfc3986(kvpair.getValue())); if (iter.hasNext()) {
buffer.append("&"); }
}
String cannoical = buffer.toString(); return cannoical;
}
/**
* Percent-encode values according the RFC 3986. The built-in Java * URLEncoder does not encode according to the RFC, so we make the * extra replacements.
*
* @param s decoded string
* @return encoded string per RFC 3986 */
private String percentEncodeRfc3986(String s) { String out;
try {
out = URLEncoder.encode(s, UTF8_CHARSET) .replace("+", "%20")
} catch (UnsupportedEncodingException e) { out = s;
}
return out; }
/**
* Takes a query string, separates the constituent name-value pairs * and stores them in a hashmap.
*
* @param queryString * @return
*/
private Map<String, String> createParameterMap(String queryString) { Map<String, String> map = new HashMap<String, String>();
String[] pairs = queryString.split("&");
for (String pair: pairs) { if (pair.length() < 1) { continue;
}
String[] tokens = pair.split("=",2); for(int j=0; j<tokens.length; j++) {
try {
tokens[j] = URLDecoder.decode(tokens[j], UTF8_CHARSET); } catch (UnsupportedEncodingException e) {
} }
switch (tokens.length) { case 1: {
if (pair.charAt(0) == '=') { map.put("", tokens[0]); } else {
map.put(tokens[0], ""); }
break; }
case 2: {
