How do you perform incremental load?

Question

I am performing SCD type 1 mapping. I have these source tables.

1.CUSTOMERS, (Source customer File)
2.PRODUCTS (Source Product File)
3.SALES   (source Sales file)

Also, these are dimensions and fact tables.

1.D_CUSTOMERS (SCD 1),
2.D_PRODUCTS (SCD 1) and
3.F_SALES_DLY (Fact)

Sales contains daily level data. Fact table will have sales & quantity data aggregated for product & for a month.(Fact table should be incremental for 24 months.(It should always contain 24 months of data)

I am bit confused, How to make fact table incremental? If i try to use any transformation(filter) to get last 24 months records, is it feasible solution?

Please suggest me something..

Thanks...

Answer 1

Incremental load is adding/inserting only changed/latest data from the source.In incremental loading,history data could remain as it is along with the new data or overwritten by incremental data.

"incremental laoding" as the name implies that the "data in source is incremented on hourly daily or monthly basis, incremental here means data is continuously added into the source", now when you implement a mapping for incremental loading then you have to run that mapping daily or on monthly basis, when you run the mapping then in that case you only wnat the current record to be fetched from source inspite of fetchin all the records. this fetching of current record and loading into the target is called incremental load.

now there are lots of method to implement the incremental load but the 2 best methods are

1.> Using Parameter file - pull the data from source system on the basis of updated dates which is taken from parameter file.

2.> using IJT(Informatica job control) - use pre session and post session SQL query in session to pull data from Source based on date from job control table. and simultanously updating the job control table with latest date.

Which one you prefer depends upon with which you are comfortable or more properly a matter of client requirments..