Problem: Load epoch timestamp into hive table.
Solution:
Solution:
- Use BIGINT and load into temp table.
- Use UDF function of to_utc_timestamp() to convert it into specific timezone.
Queries:
CREATE EXTERNAL TABLE tags_temp ( user_id INT, movie_id INT, tag STRING, date_time BIGINT ) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' COLLECTION ITEMS TERMINATED BY "|" STORED AS TEXTFILE LOCATION 'LOCATION TO THE FILE IN HDFS'; CREATE EXTERNAL TABLE tags ( user_id INT, movie_id INT, tag STRING, date_time timestamp )STORED AS ORC TBLPROPERTIES ('orc.compress'='SNAPPY','creator'='uvaraj', 'created_on' = '2016-12-30', 'description'='tags details');
INSERT OVERWRITE TABLE tags SELECT user_id,movie_id,tag,
to_utc_timestamp(date_time,'UTC') FROM tags_temp;
Comments
Post a Comment