com.gooddata.connector

Class AbstractConnector

java.lang.Object

com.gooddata.connector.AbstractConnector

All Implemented Interfaces:

Connector, Executor

Direct Known Subclasses:

ChargifyConnector, CsvConnector, DateDimensionConnector, FacebookConnector, FacebookInsightsConnector, JdbcConnector, MsDynamicsConnector, PtConnector, SfdcConnector, SugarCrmConnector

public abstract class AbstractConnector
extends Object
implements Connector

GoodData abstract connector implements functionality that can be reused in several connectors.

Version:

1.0

Author:

zd

Field Summary

Fields

Modifier and Type	Field and Description
static int	DATE_LENGTH_UNRESTRICTED
protected String	projectId Project id
protected SourceSchema	schema The LDM schema of the data source

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	AbstractConnector() Default constructor

Method Summary

Modifier and Type	Method and Description
protected void	checkLoadingStatus(String taskUri, String tmpDir, CliParams p, ProcessingContext ctx) Checks the status of data integration process in the GoodData platform
protected int	copyAndTransform(CSVReader cr, CSVWriter cw, boolean transform, int dateLength) Copies the extracted data and transform them
void	deploy(SLI sli, List<Column> columns, String dir, String archiveName) Create the GoodData data package with the ALL snapshots data
void	dump(String file) Dumps the source data CSV to a file
protected void	dumpData(Command c, CliParams p, ProcessingContext ctx) Dumps the data to CSV
static void	expandDates(SourceSchema s)
void	extract(String dir) Extracts the source data CSV to the database where it is going to be transformed
abstract void	extract(String file, boolean transform) Extract rows
void	extractAndTransfer(Command c, String pid, Connector cc, boolean waitForFinish, CliParams p, ProcessingContext ctx) Extract data from the internal database and transfer them to a GoodData project
String	generateMaqlCreate() Generates the MAQL for the data source
String	getProjectId()
Map<String,String>	getResult()
SourceSchema	getSchema() LDM schema getter
protected void	initSchema(String configFileName) Initializes the source and PDM schemas from the config file
static List<Column>	populateColumnsFromSchema(SourceSchema schema)
boolean	processCommand(Command c, CliParams cli, ProcessingContext ctx) Processes single command
protected void	setIncremental(List<Column> cols) Sets the incremental loading status for a part
protected void	setProjectId(ProcessingContext ctx) Sets the project id from context
void	setProjectId(String projectId)
void	setSchema(SourceSchema schema) LDM schema setter
protected void	transferData(Command c, CliParams p, ProcessingContext ctx) Transfers the data to GoodData project

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

schema

protected SourceSchema schema

The LDM schema of the data source

projectId

protected String projectId

Project id

DATE_LENGTH_UNRESTRICTED

public static final int DATE_LENGTH_UNRESTRICTED

See Also:

Constant Field Values

Constructor Detail

AbstractConnector

protected AbstractConnector()

Default constructor

Method Detail

getSchema

public SourceSchema getSchema()

LDM schema getter

Specified by:

getSchema in interface Connector

Returns:

LDM schema

setSchema

public void setSchema(SourceSchema schema)

LDM schema setter

Specified by:

setSchema in interface Connector

Parameters:

schema - LDM schema

extract

public void extract(String dir)
             throws IOException

Extracts the source data CSV to the database where it is going to be transformed

Specified by:

extract in interface Connector

Parameters:

dir - target directory where the data package will be stored

Throws:

IOException - in case of IO issues

dump

public void dump(String file)
          throws IOException

Dumps the source data CSV to a file

Specified by:

dump in interface Connector

Parameters:

file - target file

Throws:

IOException - in case of IO issues

copyAndTransform

protected int copyAndTransform(CSVReader cr,
                               CSVWriter cw,
                               boolean transform,
                               int dateLength)
                        throws IOException

Copies the extracted data and transform them

Parameters:

cr - - reader

cw - - writer

transform - - perform transformations?

dateLength - - cuts the fate to first dateLength chars

Returns:

number of extracted rows

Throws:

IOException

extract

public abstract void extract(String file,
                             boolean transform)
                      throws IOException

Extract rows

Parameters:

file - name of the target file

transform - perform transformations

Throws:

IOException

deploy

public void deploy(SLI sli,
                   List<Column> columns,
                   String dir,
                   String archiveName)
            throws IOException

Create the GoodData data package with the ALL snapshots data

Specified by:

deploy in interface Connector

Parameters:

sli - the SLI interface

columns - the SLI columns

dir - target directory where the data package will be stored

archiveName - the name of the target ZIP archive

Throws:

IOException - IO issues

initSchema

protected void initSchema(String configFileName)
                   throws IOException

Initializes the source and PDM schemas from the config file

Parameters:

configFileName - the config file

Throws:

IOException - in cas the config file doesn't exists

expandDates

public static void expandDates(SourceSchema s)

generateMaqlCreate

public String generateMaqlCreate()

Description copied from interface: Connector

Generates the MAQL for the data source

Specified by:

generateMaqlCreate in interface Connector

Returns:

the MAQL in string format

processCommand

public boolean processCommand(Command c,
                              CliParams cli,
                              ProcessingContext ctx)
                       throws ProcessingException

Processes single command

Specified by:

processCommand in interface Executor

Parameters:

c - command to be processed

cli - parameters (commandline params)

ctx - processing context

Returns:

true if the command has been processed, false otherwise

Throws:

ProcessingException - in case of issues during the command processing

transferData

protected void transferData(Command c,
                            CliParams p,
                            ProcessingContext ctx)
                     throws IOException,
                            InterruptedException

Transfers the data to GoodData project

Parameters:

c - command

p - cli parameters

ctx - current context

Throws:

IOException - IO issues

InterruptedException - internal problem with making file writable

dumpData

protected void dumpData(Command c,
                        CliParams p,
                        ProcessingContext ctx)
                 throws IOException,
                        InterruptedException

Dumps the data to CSV

Parameters:

c - command

p - cli parameters

ctx - current context

Throws:

IOException - IO issues

InterruptedException - internal problem with making file writable

populateColumnsFromSchema

public static List<Column> populateColumnsFromSchema(SourceSchema schema)

extractAndTransfer

public void extractAndTransfer(Command c,
                               String pid,
                               Connector cc,
                               boolean waitForFinish,
                               CliParams p,
                               ProcessingContext ctx)
                        throws IOException,
                               InterruptedException

Extract data from the internal database and transfer them to a GoodData project

Specified by:

extractAndTransfer in interface Connector

Parameters:

c - command

pid - project id

cc - connector

waitForFinish - synchronous execution flag

p - cli parameters

ctx - current context

Throws:

IOException - IO issues

InterruptedException - internal problem with making file writable

setIncremental

protected void setIncremental(List<Column> cols)

Sets the incremental loading status for a part

Parameters:

cols - SLI columns

checkLoadingStatus

protected void checkLoadingStatus(String taskUri,
                                  String tmpDir,
                                  CliParams p,
                                  ProcessingContext ctx)
                           throws InterruptedException,
                                  IOException

Checks the status of data integration process in the GoodData platform

Parameters:

taskUri - the uri where the task status is determined

tmpDir - temporary dir where the temporary data reside. This directory will be deleted.

p - cli parameters

ctx - current context

Throws:

IOException - IO issues

InterruptedException - internal problem with making file writable

getProjectId

public String getProjectId()

setProjectId

public void setProjectId(String projectId)

setProjectId

protected void setProjectId(ProcessingContext ctx)
                     throws InvalidParameterException

Sets the project id from context

Parameters:

ctx - process context

Throws:

InvalidParameterException - if the project id isn't initialized

getResult

public Map<String,String> getResult()