Data Engineering – Basic Skills – Git

Hi fellow data heroes!

Today, I want to dive into Git—a tool that’s as essential for data engineers as bash scripting. While bash scripting might make you feel like a real developer, Git takes it to the next level by facilitating collaboration and version control. After all, knowledge is only valuable if shared, and Git is the perfect tool to enable sharing and enhance our collective expertise. Remember, teamwork makes the dream work!

But, what exactly is Git?

According to Wikipedia:

Git is a distributed version control system that tracks versions of files. It is often used to control source code by programmers collaboratively developing software.

I like to think of Git as a community mural where everyone has the opportunity to contribute. Each time someone adds a new section, the mural improves. If mistakes are made, more experienced artists can correct them. The more organized the community, the more beautiful the mural becomes. Git embodies the balance between freedom and control. Just as we start with drafts, seek feedback, and refine our work before committing to the final mural, Git allows us to manage code with similar care.

Git provides a space where code evolves through contributions from multiple developers. It keeps a history of all changes, allowing you to revert to any previous version of your code. This balance between creative freedom and controlled processes is crucial for data engineers. While we should have the liberty to experiment with new techniques or enhance data pipelines, Git branches offer the structure we need. Think of branches as drafts that can be reviewed and refined before merging into the main branch. This control ensures that no change is made to the main branch without peer approval.

Git is an incredible tool for collaboration, but it’s important to establish guidelines within your team to use it effectively.

Here are some of my favorite Git commands that I use daily. While these commands are just the tip of the iceberg, they’re a great starting point for mastering Git.

I hope you find this helpful!