public class Stork extends AbstractSingleFTPerXFERJob
Stork is directly invoked by DAGMAN. The appropriate Stork modules need to be installed on the submit host.
It leads to the creation of the setup chmod jobs to the workflow, that appear as parents to compute jobs in case the transfer implementation does not preserve the X bit on the file being transferred. This is required for staging of executables as part of the workflow. The setup jobs are only added as children to the stage in jobs.
In order to use the transfer implementation implemented by this class, the
property vds.transfer.*.impl
must be set to
value Stork
.
Modifier and Type | Field and Description |
---|---|
static String |
DERIVATION_NAME
The name of the underlying derivation.
|
static String |
DERIVATION_NAMESPACE
The derivation namespace for for the transfer job.
|
static String |
DERIVATION_VERSION
The derivation version number for the transfer job.
|
static String |
DESCRIPTION
A short description of the transfer implementation.
|
static String |
TRANSFORMATION_NAME
The name of the underlying transformation that is queried for in the
Transformation Catalog.
|
static String |
TRANSFORMATION_NAMESPACE
The transformation namespace for the transfer job.
|
static String |
TRANSFORMATION_VERSION
The version number for the transfer job.
|
CHANGE_XBIT_TRANSFORMATION, mAddNodesForSettingXBit, mChmodDisabledForAllSites, mDisabledChmodSites, mLocalIrodsEnv, mLocalIrodsEnvBasename, mLocalS3cfg, mLocalS3cfgBasename, mLocalUserProxy, mLocalUserProxyBasename, mLogger, mPOptions, mProps, mRefiner, mSiteStore, mTCHandle, NOOP_PREFIX, SET_XBIT_PREFIX, XBIT_DERIVATION_NS, XBIT_DERIVATION_VERSION, XBIT_EXECUTABLE_BASENAME, XBIT_TRANSFORMATION_NS, XBIT_TRANSFORMATION_VERSION
TRANSFER_UNIVERSE, VERSION
Constructor and Description |
---|
Stork(PegasusBag bag)
The overloaded constructor, that is called by the Factory to load the
class.
|
Modifier and Type | Method and Description |
---|---|
TransferJob |
createTransferJob(Job job,
FileTransfer file,
Collection execFiles,
String txJobName,
int jobClass)
Constructs a general transfer job that handles single transfers per
transfer job.
|
boolean |
doesPreserveXBit()
Returns a boolean indicating whether the transfer protocol being used by
the implementation preserves the X Bit or not while staging.
|
protected String |
generateArgumentStringAndAssociateCredentials(TransferJob job,
FileTransfer file)
It constructs the arguments to the transfer executable that need to be passed
to the executable referred to in this transfer mode.
|
protected String |
getCompleteTCName()
Returns the complete name for the transformation.
|
protected String |
getDerivationName()
Returns the logical name of the derivation that this implementation
refers to.
|
protected String |
getDerivationNamespace()
Returns the namespace of the derivation that this implementation
refers to.
|
protected String |
getDerivationVersion()
Returns the version of the derivation that this implementation
refers to.
|
String |
getDescription()
Returns a textual description of the transfer implementation.
|
TransformationCatalogEntry |
getTransformationCatalogEntry(String siteHandle,
int jobClass)
Retrieves the transformation catalog entry for the executable that is
being used to transfer the files in the implementation.
|
boolean |
useThirdPartyTransferAlways()
Return a boolean indicating whether the transfers to be done always in
a third party transfer mode.
|
createTransferJob, createTransferJob
addSetXBitJobs, addSetXBitJobs, addSetXBitJobs, applyPriority, checkAndTransferIrodsEnvFile, checkAndTransferProxy, construct, createNoOPJob, createSetXBitJob, createSetXBitJob, createSetXBitJob, determineDisabledChmodSites, disableChmodJobCreation, getNOOPJobName, getPriority, getSetXBitJobName, setRefiner
public static final String TRANSFORMATION_NAMESPACE
public static final String TRANSFORMATION_NAME
public static final String TRANSFORMATION_VERSION
public static final String DERIVATION_NAMESPACE
public static final String DERIVATION_NAME
public static final String DERIVATION_VERSION
public static final String DESCRIPTION
public Stork(PegasusBag bag)
bag
- bag of intialization objects.public boolean useThirdPartyTransferAlways()
A value of false does not preclude third party transfers. They still can be done, by setting the property "vds.transfer.*.thirdparty.sites".
PegasusProperties.getThirdPartySites(String)
public boolean doesPreserveXBit()
public String getDescription()
public TransferJob createTransferJob(Job job, FileTransfer file, Collection execFiles, String txJobName, int jobClass)
job
- the Job object for the job, in relation to which
the transfer node is being added. Either the transfer
node can be transferring this jobs input files to
the execution pool, or transferring this job's output
files to the output pool.file
- collection of FileTransfer
objects
representing the data files and staged executables to be
transferred.execFiles
- subset collection of the files parameter, that identifies
the executable files that are being transferred.txJobName
- the name of transfer node.jobClass
- the job Class for the newly added job. Can be one of the
following:
stage-in
stage-out
inter-pool transferpublic TransformationCatalogEntry getTransformationCatalogEntry(String siteHandle, int jobClass)
siteHandle
- the handle of the site where the transformation is
to be searched.jobClass
- the job Class for the newly added job. Can be one of the
following:
stage-in
stage-out
inter-pool transfer
stage-in worker transferprotected String getDerivationNamespace()
getDerivationNamespace
in class AbstractSingleFTPerXFERJob
protected String getDerivationName()
getDerivationName
in class AbstractSingleFTPerXFERJob
protected String getDerivationVersion()
getDerivationVersion
in class AbstractSingleFTPerXFERJob
protected String generateArgumentStringAndAssociateCredentials(TransferJob job, FileTransfer file)
generateArgumentStringAndAssociateCredentials
in class AbstractSingleFTPerXFERJob
job
- the transfer job that is being created.file
- the FileTransfer that needs to be done.protected String getCompleteTCName()
getCompleteTCName
in class AbstractSingleFTPerXFERJob