Add few record(s) to a Hive table or a HDFS file on a daily basis

Question

My requirement is a typical Datawarehouse and ETL requirement. I need to accomplish

1) Daily Insert transaction records to a Hive table or a HDFS file. This table or file is not a big table ( approximately 10 records per day). I don't want to Partition the table / file.

In few articles It was being mentioned that we need to load to a staging table in Hive. And then insert like the below :

insert overwrite table finaltable select * from staging;

I am not getting this logic. How should I populate the staging table daily.

Answer 1

The staging table is typically defined as external hive table, data is loaded directly on HDFS and staging table therefore is able to read that data directly from HDFS and the transfer it to Hive managed tables, your current statement. Of course there are variations to this as well.

Answer 2

Why not INSERT INTO for appending the new data?
a) Load the new data into staging table
b) INSERT INTO final table.