$$$
{{ $t($store.state.user.experience_value_in_dollars) }}
Senior
{{ $t($store.state.user.experience_search_name) }}
0
jobs
Data Engineer
Shubham Bhardwaj
,
Lucknow, India
Experience
Other titles
Skills
I’m offering
Data Engineer seeking to leverage 8.5 years of experience in building data pipelines and wrangling big datasets. Eager to build robust data lakes that lay the groundwork for game-changing insights. Implemented natural language processing tools to make machine readable databases ready for the team of data scientists. SNAPSHOT Software Engineeri BigData ng SQL, HDFS, Spark Program Data ming Engineer Databases
Markets
United States
United Kingdom
Language
English
Fluently
Ready for
Larger project
Ongoing relation / part-time
Available
My experience
2020 - ?
job
Sr. Data Engineer
CareerBuilder.
Implemented ETL pipeline to import semi structured data
by incorporating 100 million records from 14 data sources
using pyspark and sqoop.
• Developed a tool to export data using spark and Django.
• Created and maintained Oozie workflows.
• Created a tool to parse the logs based to regex and
different formats like JSON, CSV, Grok, XML, using Spark
and Django.
• Developed a tool to import and export data from S3.
• Worked closely with data science team to extract the
text, it's cleaning to apply NLP.
• Worked on data cleaning and feature selection.
• Led a team of 5 employee, frequently collaborated with project managers and analysts.
by incorporating 100 million records from 14 data sources
using pyspark and sqoop.
• Developed a tool to export data using spark and Django.
• Created and maintained Oozie workflows.
• Created a tool to parse the logs based to regex and
different formats like JSON, CSV, Grok, XML, using Spark
and Django.
• Developed a tool to import and export data from S3.
• Worked closely with data science team to extract the
text, it's cleaning to apply NLP.
• Worked on data cleaning and feature selection.
• Led a team of 5 employee, frequently collaborated with project managers and analysts.
Data Science, JSON, XML, Django, ETL, Spark, NLP, It, Science, Pyspark, Feature, LED
2013 - 2020
job
Software Engineer
Birlasoft.
• Strong experience creating real time data streaming solutions using StreamSets.
• Work Experience in Spark, SparkSQL, Spark Streaming,
NoSQL, Hive, HDFS, Kafka, Sqoop, Oozie, Cloudera and YARN.
• Implemented hive optimized joins to gather data from different sources, used HiveQL and Impala.
• Imported and exported data using sqoop.
• Performed data transformation on data in motion.
• Work Experience in Spark, SparkSQL, Spark Streaming,
NoSQL, Hive, HDFS, Kafka, Sqoop, Oozie, Cloudera and YARN.
• Implemented hive optimized joins to gather data from different sources, used HiveQL and Impala.
• Imported and exported data using sqoop.
• Performed data transformation on data in motion.
NoSQL, Kafka, Spark, Hive, Transformation, Streaming, Software, Spark Streaming
My education
Uttar Pradesh Technical University
,
,
Lucknow Public School
,
,
Shri Ram Swaroop Memorial College of Engineering and Management
,
,
Shubham's reviews
Shubham has not received any reviews on Worksome.
Contact Shubham Bhardwaj
Worksome removes the expensive intermediaries and gives you direct contact with relevant talent.
Create a login and get the opportunity to write to Shubham directly in Worksome.
are ready to help you
and get specific bids from skilled talent in Denmark