亚洲欧美日韩高清,97超碰在线免费,国产嫩草影院久久久久

ETL时先disable外键�U�束再load数据

��翼渐丰 — Tue, 15 Jan 2008 08:12:00 GMT

Kimball在他的ETL Toolkit一书中�Q�一直强调：对于有外键约束的表，��Z��提高load的速度�Q�可以先��外键约束disable�Q�加载完成后再enable�Q�当然另外一个原因是�q�入数据仓库的数据都是规范的�Q�甚臛_��以考虑不��用外键约束，卌��反约束的数据在Transformation部分��应该解��x��Q�当然这是设计问题了�Q�。ps:另外两个提高load速度的方法分别�ؓ使用sql loader和insert、update分离。当时还不是明白�q�个�q�程怎么具体实现�Q�今天看《oracle concept》看��C��面话�l�于明白了一些：
Flexibility for Data Loads and Identification of Integrity Violations
You can disable integrity constraints temporarily so that large amounts of data can be loaded without the overhead of constraint checking. When the data load is complete,you can easily enable the integrity constraints, and you can automatically report any new rows that violate integrity constraints to a separate exceptions table.
看来oracle�q�点上做的比较完善，�q�可以将�q�反完整性约束的数据自动记录��C��个exception表中�?

��翼渐丰 2008-01-15 16:12 发表评论

在load的时候去掉sequential file的header和footer

��翼渐丰 — Mon, 13 Aug 2007 11:37:00 GMT

今天在论坛看到的�Ҏ��Q�先记录下来�Q?br>�Ҏ��一�Q?/span>
在sequential file stage中有个属性可以ignore�W�一行。对于footer�Q�如果footer使用了与正文不同的格式，可以用��用transformer stage�?@INROWNUM stage来将footer��L��?br>�Ҏ��二：
在sequential file的filter option中��用sed -e '$d' -e '1d'来讲header和footer��L��?br>�Ҏ��三：
写个before job subroutine�Q�可以先对这个文件做各种处理�Q�不�q�我想也是subroutine中调用DSExecute sed -e '$d' -e '1d'。当然Basic中也有专门对sequential file处理的函数�?br>

��翼渐丰 2007-08-13 19:37 发表评论

datastage中Orchadmin命��o的��用（转）

��翼渐丰 — Wed, 08 Aug 2007 07:37:00 GMT

Orchadmin is a command line utility provided by datastage to research on data sets.
The general callable format is : $orchadmin [options] [descriptor file]

1. Before using orchadmin, you should make sure that either the working directory or the $APT_ORCHHOME/etc contains the file “config.apt” OR

The environment variable $APT_CONFIG_FILE should be defined for your session.
Orchadmin commands

The various commands available with orchadmin are
1. CHECK: $orchadmin check

Validates the configuration file contents like , accesibility of all nodes defined in the configuration file, scratch disk definitions and accesibility of all the nodes etc. Throws an error when config file is not found or not defined properly

2. COPY : $orchadmin copy
Makes a complete copy of the datasets of source with new destination descriptor file name. Please not that

a. You cannot use UNIX cp command as it justs copies the config file to a new name. The data is not copied.

b. The new datasets will be arranged in the form of the config file that is in use but not according to the old confing file that was in use with the source.

3. DELETE : $orchadmin <> [-f -x] descriptorfiles….
The unix rm utility cannot be used to delete the datasets. The orchadmin delete or rm command should be used to delete one or more persistent data sets.
-f options makes a force delete. If some nodes are not accesible then -f forces to delete the dataset partitions from accessible nodes and leave the other partitions in inaccesible nodes as orphans.
-x forces to use the current config file to be used while deleting than the one stored in data set.

4. DESCRIBE: $orchadmin describe [options] descriptorfile.ds
This is the single most important command.
1. Without any option lists the no.of.partitions, no.of.segments, valid segments, and preserve partitioning flag details of the persistent dataset.
-c : Print the configuration file that is written in the dataset if any
-p: Lists down the partition level information.
-f: Lists down the file level information in each partition
-e: List down the segment level information .
-s: List down the meta-data schema of the information.
-v: Lists all segemnts , valid or otherwise
-l : Long listing. Equivalent to -f -p -s -v -e

5. DUMP: $orchadmin dump [options] descriptorfile.ds
The dump command is used to dump(extract) the records from the dataset.
Without any options the dump command lists down all the records starting from first record from first partition till last record in last partition.
-delim ‘’ : Uses the given string as delimtor for fields instead of space.
-field : Lists only the given field instead of all fields.
-name : List all the values preceded by field name and a colon
-n numrecs : List only the given number of records per partition.
-p period(N) : Lists every Nth record from each partition starting from first record.
-skip N: Skip the first N records from each partition.
-x : Use the current system configuration file rather than the one stored in dataset.

6. TRUNCATE: $orchadmin truncate [options] descriptorfile.ds
Without options deletes all the data(ie Segments) from the dataset.
-f: Uses force truncate. Truncate accessible segments and leave the inaccesible ones.
-x: Uses current system config file rather than the default one stored in the dataset.
-n N: Leaves the first N segments in each partition and truncates the remaining.

7. HELP: $orchadmin -help OR $orchadmin -help
Help manual about the usage of orchadmin or orchadmin commands

��翼渐丰 2007-08-08 15:37 发表评论

��翼渐丰 — Thu, 02 Aug 2007 13:01:00 GMT

当一条记录过来，某个字段为空�Q�而在transformer中又对该字段trim了，�׃��该字�D��ؓ�I�，不能trim�Q�所以DataStage会认��是个错误�Q�从而把�q�个记录drop了，�q�与实际业务不符合，非主键字�D��ؓ�I��ƈ不能��把�q�条记录drop了。我们可以通过写一个判断来解决该问题：if LK_1.EMAIL <> '' then trim(LK_1.EMAIL) else LK_1.EMAIL

��翼渐丰 2007-08-02 21:01 发表评论

DataStage中可以执行routine�Q�命令行以控制的地方

��翼渐丰 — Thu, 02 Aug 2007 11:29:00 GMT

1 before/after job和before/after stage,可以执行shell�Q�命令行以及一些DataStage API�Q�如DSJobReport
2 job sequence中有个stage�Q�可以在里面写控制job调度的地�?br>3 每个job的properties中有个job control tab
4 命��o行可以执行、导入job�{�，它通过dsjob实现�Q�在DS Manager的文档中�q�讲了有专门的命令行来导入job
5 可以�~�写shell来控制job�Q�对dsjob命��o做封装，可以传入参数�{��?br>对于上面�q�些概念�q�是很模�p�，先记录在此�?

��翼渐丰 2007-08-02 19:29 发表评论

trim全�ؓnull的某个字�D�之后，输出�?

��翼渐丰 — Thu, 02 Aug 2007 07:21:00 GMT

今天又碰到新问题�Q�在transformer中，�q�去是有很多record�Q�出来之后�ؓ0条record�Q�用peek�Q�看了一下，�q�去的数据的某个字段数据全�ؓnull�Q�而在transformer中对此字�D�trim了，改�ؓ不trim��可以了�?

��翼渐丰 2007-08-02 15:21 发表评论

��翼渐丰 — Wed, 01 Aug 2007 09:33:00 GMT

今天遇到一个问题，job不停的在那里�q�行�Q�然后link上的数据昄��各个环节都是正确的，包括最后插入数据库的link上也昄��了数据，但是最后数据库里�ƈ没有数据。在Director的log中，日志在从两个源文件把所有数据load出来完之后，日志��死在那里了�?br>以前�q�个job是正��的�Q�昨天由于重新load其中一个源文�g的元数据�Q�所以出��C��上述问题。所以先前以为是�׃��load的新的源数据有问题，��׃��此处来找问题的原因，�q�且认�ؓ可能是改了元数据�Q�在其他地方映射的时候有位置不对的地方，所以整了很久。因��Z��前是好好的，然后又以为是服务器的问题�?br>
�q�都是定势思维的错误，然后又一急，所以浪费了很多旉��Q�其实很多时候都是这��P��Z��问题我们不能理性的好好思考�?/span>

其实问题很简单：
如果我们按照正常逻辑来分析的话，既然不能��d��数据库，肯定是数据不�W�合数据库对数据的约束，包括主键啊，非空啊，本问题就是由于在stage的不断流转中产生了很多空��|��使得最后待插入的数据长度远�q�大于数据库中定义的字段长度。由于在那里不断reject�Q�所以媄响了速度�Q�job一直在那里�q�行。最后用APT_STRING_PADDER,��其设�ؓ0x0�Q�用null代替�I�格�Q�搞定�?br>ps�Q�在插入数据库时使用一个reject文�g�Ҏ��错有好处�Q�这栯��看到reject是些什么数据，然后��p��知道��Z��么被reject了�?br>同时我们可以得出如果最后插入数据库时很多数据被reject�Q�但是你�q�没有用一个reject文�g来接收这些reject掉的数据�Q�将使得job基本处于停滞状态�?/span>

��翼渐丰 2007-08-01 17:33 发表评论

��翼渐丰 — Wed, 01 Aug 2007 02:17:00 GMT

当一个Job跑到一半终止了要还原，在DS Director中主要有三种�Ҏ��来还原：
1 Job-----Reset
2 Job-----Clear Status File
3 Job-----Cleanup Resource
具体�l�节也不懂，有时间慢慢研�I?/p>

��翼渐丰 2007-08-01 10:17 发表评论

��翼渐丰 — Tue, 24 Jul 2007 09:00:00 GMT

Datastage Desiner的快捷键
1 关闭某个job
2 同时打开了两个以上job�Q�在不同的job之间切换�?br>Datastage Director的快捷键

��翼渐丰 2007-07-24 17:00 发表评论

调用dsjob的shell script

��翼渐丰 — Wed, 18 Jul 2007 02:32:00 GMT

#!/bin/ksh
#######
################################################################################
#######
####### FILE: MasterControl.ksh
#######
####### DESCRIPTION: Starts a DataStage MasterControl type job passing
####### all runtime parameter values
#######
#######
####### Date Version Developer Description
####### ---------- --------- -------------- ------------------------------------
####### 2002-05-15 1.0 Ken Bland Initial release
####### 2002-06-27 2.2 Ken Bland FileSetDate/ProcessDate modifications
####### 2002-07-10 2.2 Steve Boyce Added ProcessDate as 14th parameter
####### 2002-08-16 2.3 Steve Boyce Now calls stored procedure
####### GET_NEW_BATCH_NBR in datastage schema
####### instead of deriving it and inserting
####### here.
####### Uses comSQLPlus.ksh and comPLSQL.ksh
####### instead of SQLPlusStub.ksh.ksh
#######

PROG=`basename ${0}`
EXIT_STATUS=0

NOW=`date`
echo "${NOW} ${PROG} Initialization..."
echo

#######
####### CONFIGURATION ##########################################################
#######

if [ ${#} -ne 14 ]; then
echo "${NOW} ${PROG} : Invalid parameter list."
echo "${NOW} ${PROG} : The script needs 14 parameters:"
echo "${NOW} ${PROG} : JobName"
echo "${NOW} ${PROG} : ParameterFile"
echo "${NOW} ${PROG} : FileSetDate (YYYY-MM-DD)"
echo "${NOW} ${PROG} : BatchNumber"
echo "${NOW} ${PROG} : JobHierarchyFile"
echo "${NOW} ${PROG} : SourceSystemList"
echo "${NOW} ${PROG} : SubjectAreaList"
echo "${NOW} ${PROG} : ClearWorkArea"
echo "${NOW} ${PROG} : StartingMilestone"
echo "${NOW} ${PROG} : EndingMilestone"
echo "${NOW} ${PROG} : DebugMode"
echo "${NOW} ${PROG} : JobLinkStatisticChecksFile"
echo "${NOW} ${PROG} : ResurrectLogFile"
echo "${NOW} ${PROG} : ProcessDate (NULL|YYYY-MM-DD H24:MI:SS)"
exit 99
fi

JobName="${1}"
ParameterFile="${2}"
FileSetDate="${3}"
BatchNumber="${4}"
JobHierarchyFile="${5}"
SourceSystemList="${6}"
SubjectAreaList="${7}"
ClearWorkArea="${8}"
StartingMilestone="${9}"
EndingMilestone="${10}"
DebugMode="${11}"
JobLinkStatisticChecksFile="${12}"
ResurrectLogFile="${13}"
ProcessDate="${14}"

echo "${NOW} ${PROG} JobName ${JobName}"
echo "${NOW} ${PROG} ParameterFile ${ParameterFile}"
echo "${NOW} ${PROG} FileSetDate ${FileSetDate}"
echo "${NOW} ${PROG} BatchNumber ${BatchNumber}"
echo "${NOW} ${PROG} JobHierarchyFile ${JobHierarchyFile}"
echo "${NOW} ${PROG} SourceSystemList ${SourceSystemList}"
echo "${NOW} ${PROG} SubjectAreaList ${SubjectAreaList}"
echo "${NOW} ${PROG} ClearWorkArea ${ClearWorkArea}"
echo "${NOW} ${PROG} StartingMilestone ${StartingMilestone}"
echo "${NOW} ${PROG} EndingMilestone ${EndingMilestone}"
echo "${NOW} ${PROG} DebugMode ${DebugMode}"
echo "${NOW} ${PROG} JobLinkStatisticChecksFile ${JobLinkStatisticChecksFile}"
echo "${NOW} ${PROG} ResurrectLogFile ${ResurrectLogFile}"
echo "${NOW} ${PROG} ProcessDate ${ProcessDate}"
echo

# Below will look in the parameters.ini file to determine the directory path each.
UserID=`whoami`
BinFileDirectory=`cat /.dshome`/bin
LogFileDirectory=`grep -w LogFileDirectory ${ParameterFile}|cut -d "=" -f2`
TempFileDirectory=`grep -w TempFileDirectory ${ParameterFile}|cut -d "=" -f2`
CommonScriptFileDirectory=`grep -w CommonScriptFileDirectory ${ParameterFile}|cut -d "=" -f2`
CommonLogFileDirectory=`grep -w CommonLogFileDirectory ${ParameterFile}|cut -d "=" -f2`
LogFileName=${CommonLogFileDirectory}/${PROG}_${JobName}.log
TEMPBATCHNBRLOG=${TempFileDirectory}/${PROG}_${JobName}_start.log
DATASTAGEPROJECT=`grep -w DATASTAGEPROJECT ${ParameterFile}|cut -d "=" -f2`
DSSERVER=`grep -w DSSERVER ${ParameterFile}|cut -d "=" -f2`
DSUSERID=`grep -w DSUSERID ${ParameterFile}|cut -d "=" -f2`
DSPASSWORD=`grep -w DSPASSWORD ${ParameterFile}|cut -d "=" -f2`

NOW=`date`
echo "${NOW} ${PROG} UserID ${UserID}"
echo "${NOW} ${PROG} BinFileDirectory ${BinFileDirectory}"
echo "${NOW} ${PROG} LogFileDirectory ${LogFileDirectory}"
echo "${NOW} ${PROG} TempFileDirectory ${TempFileDirectory}"
echo "${NOW} ${PROG} CommonScriptFileDirectory ${CommonScriptFileDirectory}"
echo "${NOW} ${PROG} CommonLogFileDirectory ${CommonLogFileDirectory}"
echo "${NOW} ${PROG} LogFileName ${LogFileName}"
echo "${NOW} ${PROG} TEMPBATCHNBRLOG ${TEMPBATCHNBRLOG}"
echo "${NOW} ${PROG} DATASTAGEPROJECT ${DATASTAGEPROJECT}"
echo "${NOW} ${PROG} DSSERVER ${DSSERVER}"
echo "${NOW} ${PROG} DSUSERID ${DSUSERID}"
echo "${NOW} ${PROG} DSPASSWORD *Protected*"
echo

#######
####### PARAMETER BUILD Without batch number ##################################
#######

if [ "${ProcessDate}" = "NULL" ]; then
StartTimestamp=`date '+%Y-%m-%d %H:%M:%S'`
else
StartTimestamp="${ProcessDate}"
fi
ParamList=" -param ParameterFile=${ParameterFile}"
ParamList="${ParamList} -param ProcessDate=\"${StartTimestamp}\""
ParamList="${ParamList} -param FileSetDate=${FileSetDate}"
ParamList="${ParamList} -param JobHierarchyFile=${JobHierarchyFile}"
ParamList="${ParamList} -param SourceSystemList=${SourceSystemList}"
ParamList="${ParamList} -param SubjectAreaList=${SubjectAreaList}"
ParamList="${ParamList} -param ClearWorkArea=${ClearWorkArea}"
ParamList="${ParamList} -param StartingMilestone=${StartingMilestone}"
ParamList="${ParamList} -param EndingMilestone=${EndingMilestone}"
ParamList="${ParamList} -param DebugMode=${DebugMode}"
ParamList="${ParamList} -param JobLinkStatisticChecksFile=${JobLinkStatisticChecksFile}"
ParamList="${ParamList} -param ResurrectLogFile=${ResurrectLogFile}"

#######
####### Get Batch Number and create ETL_BATCH_AUDIT record #####################
#######

echo "${NOW} ${PROG} About to get new BATCH_NBR and insert it into ETL_BATCH_AUDIT..."
${CommonScriptFileDirectory}/comPLSQL.ksh ${ParameterFile} "IRDSN" "IRUserID" \
GET_NEW_BATCH_NBR \
"${JobName}" \
"${StartTimestamp}" \
"${UserID}" \
"${SourceSystemList}" \
"${SubjectAreaList}" \
"${ParamList}" \
"${FileSetDate}" > ${TEMPBATCHNBRLOG}
SQL_EXIT_STATUS=$?
cat ${TEMPBATCHNBRLOG}
if [ "${SQL_EXIT_STATUS}" != 0 ]; then
NOW=`date`
echo "${NOW} ${PROG} Failure to connect/insert into ETL_Batch_Audit table!"
exit ${SQL_EXIT_STATUS}
fi

#######
####### Get BATCH_NBR from batch number log file ##############################
#######
BatchNumber=`grep -w BATCH_NBR ${TEMPBATCHNBRLOG}|cut -d "=" -f2`
if [ -z "${BatchNumber}" ]; then
NOW=`date`
echo "${NOW} ${PROG} Failure to retrieve BATCH_NBR from ${TEMPBATCHNBRLOG}"
exit ${SQL_EXIT_STATUS}
fi

#######
####### Add batch number to list of parameters #################################
#######
ParamList="${ParamList} -param BatchNumber=${BatchNumber}"

NOW=`date`
echo
echo ${NOW} ${PROG} Parameter list: ${ParamList}
echo

#######
####### DataStage EXECUTION ####################################################
#######

NOW=`date`
echo "${NOW} ${PROG} Executing DataStage dsjob program..."

echo ${BinFileDirectory}/dsjob -server ${DSSERVER} -user ${DSUSERID} -password ${DSPASSWORD} -run -wait ${ParamList} ${DATASTAGEPROJECT} ${JobName} 2>&1 > ${LogFileName}
echo
echo "${BinFileDirectory}/dsjob -server ${DSSERVER} -user ${DSUSERID} -password ${DSPASSWORD} -run -wait ${ParamList} ${DATASTAGEPROJECT} ${JobName} 2>&1 > ${LogFileName}"
echo

eval ${BinFileDirectory}/dsjob -server ${DSSERVER} -user ${DSUSERID} -password ${DSPASSWORD} -run -wait ${ParamList} ${DATASTAGEPROJECT} ${JobName} 2>&1 >> ${LogFileName}

jobwaiting=`grep "Waiting for job..." ${LogFileName}`
if [ "${jobwaiting}" != "Waiting for job..." ]; then
NOW=`date`
echo ${NOW} ${PROG} "DataStage failed to start the job"
failedstart=1
else
NOW=`date`
echo ${NOW} ${PROG} "DataStage successfully started the job"
failedstart=0
fi
NOW=`date`
echo ${NOW} ${PROG} "Retrieving job information"

${BinFileDirectory}/dsjob -server ${DSSERVER} -user ${DSUSERID} -password ${DSPASSWORD} -jobinfo ${DATASTAGEPROJECT} ${JobName} >> ${LogFileName}

#######
####### CHECK STATUS ###########################################################
#######

ERROR=`grep "Job Status" ${LogFileName}`
ERROR=${ERROR##*$}
ERROR=${ERROR%%$*}

if [ "${failedstart}" != 0 ]; then
NOW=`date`
echo ${NOW} ${PROG} "The job failed to start"
AuditStatus="FAILURE"
Comments="MasterControl aborted"
EXIT_STATUS=1
else
if [ "${ERROR}" = 1 -o "${ERROR}" = 2 ]; then
NOW=`date`
echo ${NOW} ${PROG} "The job completed successfully"
AuditStatus="SUCCESS"
Comments=""
EXIT_STATUS=0
else
NOW=`date`
echo ${NOW} ${PROG} "The job aborted"
AuditStatus="FAILURE"
Comments="MasterControl aborted"
EXIT_STATUS=1
fi
fi

FailedJobCount=`grep -i FAILED ${LogFileDirectory}/${JobName}.log|wc -l|cut -b1-9`
FailedJobCount=`expr ${FailedJobCount} + 0`
echo ${NOW} ${PROG} The number of failed jobs is [${FailedJobCount}]

if [ "${FailedJobCount}" != 0 ]; then
NOW=`date`
echo ${NOW} ${PROG} "The job had failed processes"
AuditStatus="FAILURE"
Comments="MasterControl had ${FailedJobCount} failed processes"
EXIT_STATUS=1
fi
StoppedJobStreamCount=`grep "JOB STREAM STOPPED" ${LogFileDirectory}/${JobName}.his|wc -l|cut -b1-9`
StoppedJobStreamCount=`expr ${StoppedJobStreamCount} + 0`
if [ "${StoppedJobStreamCount}" != 0 ]; then
NOW=`date`
echo ${NOW} ${PROG} "The job stream was STOPped or KILLed"
AuditStatus="FAILURE"
Comments="MasterControl job stream was STOPped or KILLed"
EXIT_STATUS=1
fi

#######
####### AUDIT ##################################################################
#######

echo
echo "${NOW} ${PROG} About to update ETL_BATCH_AUDIT with status information..."
EndTimestamp=`date '+%Y-%m-%d %H:%M:%S'`

SQLstring="UPDATE ETL_BATCH_AUDIT A \
SET A.END_TIMESTAMP = TO_DATE('${EndTimestamp}','YYYY-MM-DD HH24:MI:SS'), \
A.STATUS = '${AuditStatus}', \
A.COMMENTS = '${Comments}', \
A.RUNTIMESETTINGS = '${ParamList}' \
WHERE (A.BATCH_NBR = ${BatchNumber});"

NOW=`date`
echo ${NOW} ${PROG} Audit SQL ${SQLstring}
SQLScriptFileName=${TempFileDirectory}/${PROG}_${JobName}_end.sql
echo ${SQLstring} > ${SQLScriptFileName}

${CommonScriptFileDirectory}/comSQLPlus.ksh ${ParameterFile} IRDSN IRUserID ${SQLScriptFileName}
SQL_EXIT_STATUS=$?
if [ "${SQL_EXIT_STATUS}" != 0 ]; then
NOW=`date`
echo ${NOW} ${PROG} Failure to connect/update into ETL_Batch_Audit table!
exit ${SQL_EXIT_STATUS}
fi

#######
####### EXIT ###################################################################
#######

NOW=`date`
echo ${NOW} ${PROG} Complete, exiting with status [${EXIT_STATUS}]
exit ${EXIT_STATUS}

好东西，先收着�Q�再研究�?/span>

��翼渐丰 2007-07-18 10:32 发表评论

DataStage中不能删除Job

��翼渐丰 — Sat, 14 Jul 2007 06:34:00 GMT

在DataStage中job不能删除�Q�如果试着�~�译又出现如下错误：
Error calling subroutine: DSR_EXECJOB (Action=1); check DataStage is set up correctly in project ETUDES
(Subroutine failed to complete successfully (30107))

(40503) A call to an OLE server has failed, or a runtime error occured within the ole itself.

解决办法�Q?br>1 首先�q�行命��o�Q�DELETE FROM DS_JOBS WHERE NAME = 'JobName'
2 再运行DS.CHECKER �Q�注意要�q�行DS.CHECKER必须要有exclusive access到project
�Q�通过如下命��o可以看到有谁�q�连接到了服务器�Q�netstat -a |grep dsrpc

��翼渐丰 2007-07-14 14:34 发表评论

DataStage Basic学习�W�记

��翼渐丰 — Thu, 12 Jul 2007 06:46:00 GMT

一 BASIC介绍

1 一般的BASIC�E�序是如下一个格�?br>[ label ] statement [ ; statement … ]
2 关于subroutine
a 如果嵌入��C��E�序中用GOSUB调用�Q�如果在外部�Q��用CALL
b 使用return�q�回��C��E�序�Q�如果在return之前subroutine�l�束了，自动return
c 如果subroutine中有ABORT STOP CHAIN则永�q�回不到�ȝ��序�?br>3 注释
以REM * ! $^*开头的为注�?br>4 创徏一个BASIC �E�序的步�?br>使用CREATE.FILE命��o创徏一个type 1或type 19 DataStage file来保存BASIC�E�序的源码�?在DSEngine目录下运行CREATE.FILE命��o�Q�它��提�C�Z��输入文�g名等参数)

使用DataStage Editor或其他编辑器创徏BASIC�E�序源码�Q�这个源码成��Z��刚才创徏的文件的一条记录�?/p>

使用BASIC命��o来编译你�E�序�Q�这��创��Z��个包含目标代码的文�g。目标文件的名字不重要，因�ؓ��L��通过源文件名来引用程序�?/p>

可以用RUN命��o来执行你的程序，使用RAID命��o来调试你的程序�?/p>

二、基��知识
数据�c�d��、变量和操作�W?br>所有数据在内部都是以字�W�串存储�Q�在�q�行时�{换�ؓ具体�c�d��?br>主要有三�U�数据类型：字符丌Ӏ�数字型、和unknown�Q�null�Q?/p>

字符�?br>在BASIC源码中，字符串��量是ASCII字符的序列，所以不能有0�Q�null�Q�和10�Q�linefeed�Q�字�W�，而是要��用CHAR(0)和CHAR(10)
数字�?br>范围�?0(-307)�?0(307)�Q�最�?5位小�?br>unknown
null在内部是由单一字节CHAR(128)代表的，是一个string�Q�可以用�pȝ��变量@NULL.STR来引用它。在BASIC�E�序中可以��用系�l�变量@NULL来引用null倹{�?br>一个字�W�串与null�q�接�l�果为null

数组
dimensioned array
在向量（一�l�数�l�）中，�W�一个元素的index�?
向量可以看做�W�二位�ؓ1的二�l�数�l�（矩阵�Q��?br>矩阵和向量的�W�一个元素分别用vector.name(0)和matrix.name(0,0)表示。它可以在MATPARSE、MATREAD和MATWRITE语句中用来存储不合格的字�D�c�?/p>

动态数�l?br>动态数�l�的最高层是字�D�，�?F)分割。字�D�包含有(V)分割的值组成。每个值又可以包含�?S)分割的subvalue�?/p>

字符串的比较
字符串的比较是一个个的比较每个字�W�的ASCII的倹{��如果两个字�W�串可以转换为数字，��L��比较两个数字的数值大��。�Q何��g��null相比较的�l�果都是unknown。空串不是简单的�{�于0。空格看做小�?�?/p>

字符串匹�?br>X代表��M��字符 A代表所有字�?N代表所有数�?前面的数字代表重复次数。如果前面的数字�?�Q�代表后面的字符重复0到N�ơ的��L��ơ数都可以�?/p>

0是false �I�Z��是false null非true非false

�?事务和隔��ȝ��?br>�?br>锁有两种�_�度�Q�细�_�度�Q�记录锁定）和粗�_�度�Q�文仉��定）
DataStage支持以下几种锁：
记录�׃�n�?br>记录更新�?br>文�g�׃�n�?br>文�g意向�Q�intent�Q�锁介于�׃�n与排他之�?br>文�g排他�?/p>

事务
事务可以嵌套
子事务在父事务提交或回滚之前开始的话，子事务将变成�z�d��事务�Q�而父事务��l�存在但是是非活动的。当子事务提交或回滚之后�Q�父事务��再�ơ变为活动的�?br>在最高层的事务提交之前，��缓存所有子事务的数据库操作。当�q�行一个读取操作时�Q�首先会去读取缓存的数据�Q�如果找不到才会从数据库文�g中读取�?br>

��翼渐丰 2007-07-12 14:46 发表评论

在Java中调用DataStage的job

��翼渐丰 — Tue, 10 Jul 2007 12:13:00 GMT

论坛上见到的�Q�记录一下，以备後用
Process p = Runtime.getRuntime().exec( MProperties.DS_EXE_PATH + "/dsjob -run -wait -param lowDate=" + lowDate + " -param highDate=" + highDatePlusOne + " -param dsOutDir=" + MProperties.DS_OUT_DIR + " -param IDB_user=" +MProperties.IDB_USER + " -param IDB_pass=" +
MProperties.IDB_PASS + " " + MProperties.DS_PROJECT + " " + MProperties.DS_JOB);
p.waitFor();
return p.exitValue() == 0;

另外如果有RTI�Q�更好的�Ҏ��如下�Q?/span>
The best way for this to be accomplished is by using the RTI (Real Time Integration) capabilities that we added in 7.x. This allows you to fully expose a DataStage job as a service, to be then invoked via several different mechanisms, depending on your needs....

.....the job can be exposed as a classic web service, using SOAP over HTTP...we generate and publish the WSDL for this within RTI...

....the job can be exposed via our EJB binding and called appropriately

....the job can be exposed for contact via JMS, and either passed a text message or SOAP envelope....

Not only can you simply pass job parameters via this architecture, you can actually have the entire transformation job itself (columns on the ultimate input and output links) passed and returned to your calling client.

Additionally, the entire job control API, which you may already be familiar with in its API or command line implementations, has been exposed as a set of methods available via Web Services.
主要��是通过��job包装为web service或EJB或JMS来调用，可以传参敎ͼ��q�可以将整个job本��n�Q�最�l�的输入和输出链接）作�ؓ参数传递或者返回到调用的客��L��?

��翼渐丰 2007-07-10 20:13 发表评论

DataStage官方文档学习�W�记

��翼渐丰 — Thu, 28 Jun 2007 01:43:00 GMT

通过右键��d��link
鼠标右键点击起始stage�Q�按住右键移动鼠标到目标stage。还有一�U�方法就是把鼠标攑֜�起始stage的边�~�等到变��Z��个小圆圈里面有一个叉的时候拖动鼠标到目标stage�?br>
DataStage中默认和隐式�c�d��转换时注意的问题
当从源向目标映射数据�Ӟ��如果�c�d��不一��_��对于有些�c�d��我们需要在modify或transfomer stage中通过函数�q�行转换�Q�对于有些系�l�会自动完成�c�d��转换�Q�在�c�d��转换�q�程中，注意以下几点�Q?br>1 在变长到固定长度字符串的默认转换中，parallel jobs用空��|��ASCII 20�Q�字�W�来填充剩余长度。联�p�d��前面做的一个parallel job�Q�当目标是变长时�Q�当时源也是变长�Q�但是好像源的精度要��些�Q�，但是字符串实际的长度没有指定的变镉K��么长的话�Q�系�l�会自动用空��|��ASCII 20�Q�来填充�Q�具体在哪个stage填充的不知）�Q�而且环境变量APT_STRING_PADCHAR的默认��g��是空��|��ASCII 20�Q��?br>2 通过环境变量APT_STRING_PADCHAR可以改变默认的填充字�W�null�Q�ASCII 0�Q�，同时一般还要加上trim��p��解决�I�格问题�Q�此问题非常普遍�?br>3 有个PadString 函数可以用来用指定的字符来填充一个变长的字符串到指定的长度。这个函数的参数不能使固定长度字�W�串�Q�如果是固定长度的先转化为变�ѝ�?br>
Copy Stage作�ؓ占位�W?/strong>
在job开发中�Q�当你暂时不知道该��用哪个stage�Ӟ��可以使用copy stage作�ؓ占位�W�，注意不要把Force属性设为True�Q�在不把Force设�ؓTrue�Ӟ��复制��在�q�行时进行优化�?br>
使用Schema file来指定meta data
1 可以通过列定义和schema file两种方式来指定meta data�Q�值得注意的一�Ҏ��Note that, if you use a schema file, you should ensure that runtime column propagation is turned on. Otherwise the column definitions specified in the stage editor will always override any schema file.卻I��如果想通过schema file来指定meta data�Q�必��d��上runtime column propagation�Q�否则��L��使用列定义来作�ؓmeta data�?
2 If you are using a schema file on an NLS system, the schema file needs to be in UTF-8 format.如果你的datastage使用了NLS system�Q�那么schema file必须是UTF-8格式的。文档中说可以��用Sequential stage来完成�{码�?br>
通过Server Shared Container在Parallel Job中添加Server Job Stage
You create a new shared container in the DataStage Designer, add Server job stages as required, and then add the Server Shared Container to your Parallel job and connect it to the Parallel stages.在Designer中创��Z��个server shared container�Q�添加需要的Server job stage�Q�再把server shared container��d��到parallel job�q�连接到其他parallel stage�?br>
几个环境变量的具体�\�?/span>
%SERVERENGINE% – DataStage engine account directory(normally C:\Ascential\DataStage\Engine on Windows and \Ascential\DataStage\DSEngine on UNIX).
%PROJECT% – Current project directory.
%SYSTEM% – System directory on Windows NT or /usr/lib on UNIX.

一句不太懂的话�Q�先做记�?/span>
If NLS is enabled, avoid using the built-in Iconv and Oconv functions to map data unless you fully understand the consequences of your actions.在��用了NLS�Ӟ��量避免使用Iconv and Oconv来映��日期，因�ؓ可能产生一些意想不到的�l�果�?br>

��翼渐丰 2007-06-28 09:43 发表评论