Master of Data Science and Analytics has become quite well implemented as part of universities’ curriculums. They are now over a thousand masters offered across different departments and schools, from schools of Computer Science, Mathematics & Statistics, Economics or Business. When considering a MSc in Data Science, a few questions…

An overview of the different profiles of Data Engineers — With the increased digitalization and data use cases stemming from it, the field of data engineering is becoming quite in demand. Yet more often than not, hiring managers and companies don't fully grasp the nuance of the field. …

Data Engineers need to understand the different files format they need to read from and write to, as well as the limitations and pitfalls of each format. …

How Cython or Nim can help you get more from Python and achieve better performance — There are many ways to speed up Python, from writing more efficient python code to leveraging optimized libraries such as Numpy when doing mathematical computation. …

How to leverage MLOps principles for more impactful machine learning initiatives — A couple of years ago, I touched base on some of the key principles to handling the deployments of machine learning models. These days merely deploying machine learning models into production is often not sufficient. What matters is to operate them at scale, and this requires a very different process…

Typically, Pandas find its' sweet spot in usage in low- to medium-sized datasets up to a few million rows. Beyond this, more distributed frameworks such as Spark or Dask are usually preferred. It is, however, possible to scale pandas much beyond this point. The typical issue with scaling with Pandas…

It is essential to do code reviews in Data Engineering. Code reviews provide a good foundation for the future, especially when looking at real-time use cases and a way to avoid regressions now. Data Engineering code review can be similar to code reviews for Software engineering, but since Data Engineering…

Financial methods, applications, and modeling for Data Engineers — Being a role meant to support the decision-making process, Data Engineers need to understand certain Financial concepts and know-how to best leverage them in their data models. Some concepts are particularly important for Data Engineers activities, namely amortization, and allocation. While other concepts of controlling such as entitlement values, mix…

Data Engineering application of data classes — Data classes are a relatively new introduction to Python, first released in Python 3.7 which provides an abstraction layer leveraging type annotations to define container objects for data. Compared to a normal Python class, data classes make do of some of the syntactic sugar for instantiation, and there are a…