ON DATA ENGINEERING

Photo by Markus Spiske on Unsplash

Being a role meant to support the decision-making process, Data Engineers need to understand certain Financial concepts and know-how to best leverage them in their data models.

Some concepts are particularly important for Data Engineers activities, namely amortization, and allocation. While other concepts of controlling such as entitlement values, mix…


ON Data Engineering

Photo by Nam Hoang on Unsplash

Data classes are a relatively new introduction to Python, first released in Python 3.7 which provides an abstraction layer leveraging type annotations to define container objects for data. …


ON DATA ENGINEERING

Photo by Artur Shamsutdinov on Unsplash

DBT is a tool that aims at facilitating the work of analysts and data engineering in transforming data and modeling within a data warehouse. It provides a command-line as well as a documentation and RPC server.

After more than a year working with DBT, I thought it would be good…


Photo by Michael Dziedzic on Unsplash

Data Modeling seems to have become a lost art amongst data engineers. What was once the primal part of the job of a data engineer seems to have been relegated to a secondary rank.

Shaping the data by developing an understanding of the underlying data and the business process going…


Photo by Markus Winkler on Unsplash

This month was quite active with news and release in the Data space, with two big conferences going on — Amazon’s re:Invent and Neurips, as well as the official release of Airflow 2.0 and an introduction to the principles and architecture of the Data Mesh…

SQL, Databases, and ETL

SQLite received a new release(3.34.0)


ON DATA ENGINEERING

Photo by Robin Pierre on Unsplash

The shift towards real-time data flow has a major impact on the way applications are designed and on the work of data engineers. Dealing with real-time data flows brings a paradigm shift and an added layer of complexity compared to traditional integration and processing methods (i.e., batch).

There are real…


Photo by Viacheslav Bublyk on Unsplash

There are different estimates for Salaries available online. Glassdoor provides a general overview of salaries for Data Scientists, StackOverflow provides a specific calculator, and several recruitment agencies provide salary estimates for different positions.

For instance, Harnham produces an annual salary guide for many data professions across the UK, several European…


Photo by Markus Winkler on Unsplash

With everyone trying to get everything out of the door before the Holiday season, November was a busy month for the data world, Airflow 2.0 …


ON DATA ENGINEERING

Photo by abi ismail on Unsplash

Postgres as a database is a very versatile database, with a high degree of extensibility. It can be extended through extensions, UDFs, UDAF, UDT. There are quite a few features not currently available within the native implementation. …


ON DATA ENGINEERING

Photo by Campaign Creators on Unsplash

SQL is one of the key tools used by data engineers to model business logic, extract key performance metrics, and create reusable data structures. There are, however, different types of SQL to consider for data engineers: Basic, Advanced Modelling, Efficient, Big Data, and Programmatic. …

Julien Kervizic

Living at the interstice of business, data and technology | Head of Data | iptiQ by SwissRe, Facebook and Amazon | linkedin: https://bit.ly/2XbDffo

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store