I am someone who always says “it depends” about many things, especially at work and about technology. It depends because it is so, and I do not trust those who have the answer ready. But saying “it depends” on many things does not mean saying it on everything. There are some things where my position is clear and sharp, and for me it is not up for discussion. You can discuss it, of course, and I leave it to the people who read to judge.
This is one of those things. You do data analysis? Ok, then you must know SQL. “No, but I am a data scientist.” It does not matter. “No, but so far I did not need it.” Look, let us put it this way: you do not want to become good, ok? You are not interested in the job, you have no passion for it, fine. But if you tell me that you want to become a good data analyst, you must know SQL. Period.
You are using the same reasoning as someone who says, “I want to travel a lot, I want to see a lot of places, I want to know new cultures, but I do not want to learn English.” Fine, you can do it, but you will only go where they speak your language. You are limiting yourself, you are cutting your own hands. You are making a totally inconsistent choice.
You should know SQL and data modeling. Why? Because if you do not know them, you do not understand some fundamental things that do not depend on the language. The AI writes the language anyway, so who cares about the syntax?
“Knowing SQL” means that you know the concepts behind the syntax of the language. For example: NULL and zero are two different things; data is structured and organized in tables; different tables have logical relationships; the content of different tables can be combined (joined) by using these logical relationships and other constraints; if a relationship does not have a unique column on one side, the result of the join can have duplicated rows that increase the aggregated total; the result of a query can be used for other queries, and so on.
Every few years, a new format comes out, and then articles appear that say, “If you want to optimize, you must do this.” I read them, and I think: we went back 20 years. They invent a new format, and they’re facing the same problems relational databases faced over 50 years ago. Maybe they are doing it faster, but they are facing the same problems, and they are solving them with the same techniques, until they arrive exactly at the point where relational databases were. It makes me laugh, because whoever writes these articles genuinely thinks he discovered something incredible. My friend, you just arrived. It has been like this for a lifetime, and we already did it. At a conference, to explain this, I asked a sensitive question: how many of you have seen, or remember, defrag? Half of the audience raised their hand. The other half had no idea what it means.
English, by the way, is not a random example. I am Italian, and more than once I have been told that my content should be in Italian. Just because I am Italian? I can’t produce this content in Italian; there would be no market, it wouldn’t be economically sustainable, so I have to do it in English. We can speak Italian when we watch the football match. At work, we work in English because the market is international. Period. If you prefer to speak your dialect and limit your market to your small town, you are free to do it, but then don’t complain, and above all don’t go around saying “why don’t you speak my dialect?”. Hey, there are 4,000 dialects in Italy; I cannot speak them all.
And if you are in a school where English is discouraged, run away immediately; without any doubt, you are in the wrong place. There is nothing else to evaluate. Run. Your professor told you that English is not important? Run away immediately. It doesn’t matter if he was joking; it doesn’t matter. You must run, because your future is in extreme danger, and at the moment when you should be learning, they are forbidding it. There is only one thing to do. Run. There are no others.
SQL for data is what English is for travelers and tourists. You can live without it, but you limit yourself by doing so.
“The AI writes the SQL now.”
Yes. We are doing a course on AI-assisted development with DAX, and we are seeing that the competence of the latest models is above that of most students at the end of our course. We tested the simpler problems first, but even where there is a whole path of reasoning to follow, it does it. A whole line of people keep saying the AI repeats what it has read. Sorry, no. It is not like that anymore. There is a capacity to generalize concepts and to apply them to a different problem that was not there two years ago, and six months ago it was not so sophisticated. And there is the capacity to work for one or two hours without going off the rails. Obviously, some things still don’t work, but belittling what is there would be stupid. Ok, let’s take note; let’s move on.
If you create a report with your name on it, you are responsible. If the AI wrote your query, you are signing it anyway. But if you do not read the query generated by the AI, you are just forwarding a report created by something else. Are you still necessary?
This is the difference between those who follow instructions and those who think for themselves. If you only follow instructions, you are a chatbot like any other. Now that we have actual chatbots, you are replaceable. What surprises Alberto and me in the work we are doing is when the LLM goes one mile further: it doesn’t limit itself to doing what you asked; it finds inconsistencies between the information you gave it and what you asked.
For example, this week I did this job. We built a telemetry system for some internal work, and I wanted to analyze its data. I said: this is the description of the data, here is the documentation, I want to do this analysis, do this, this, and this. It started by making a plan, and I told it, “No, this is wrong; we do it like this, we do it like that”. It had some biases, and I fixed them. When it understood how things should be done, it started doing them, and at some point it said: “I looked at the data, I looked at the documentation, and the data does not match the documentation. There are these discrepancies. I made a document with all the discrepancies, so if you want to update the documentation, this is the document to use.” I honestly did not even open the document. I said, " Let’s see if it’s right, and I sent it to the person who wrote the documentation: “Is this stuff right?” “Yes, it’s right. I had not updated this; I will fix it.” Ok, now the documentation is fixed; this is the updated version. Review everything again.
If the LLM starts doing the job I expect from a human, the bar for you as a human went up even more. And the guy who says, “Hi, I introduce myself: you tell me what to do, you give me the salary”, you have a problem. You have a serious problem, because you are the most replaceable person on the planet, with something that costs less.
We wrote it in the SQLBI guidelines on generative AI: use AI to improve your productivity; don’t become a victim of AI. If you stop thinking, you become disposable. Your added value is applying the 5 to 10% of corrections needed to the output. You’re still needed, but you must think. We are not against AI. We are saying that you must keep a critical eye, and if you stop thinking, you are no longer necessary.
For example, if a query has a misplaced join, it could produce duplicated data in an aggregate total. You may see a reasonable aggregation, but the underlying query could join tables incorrectly, and if the numbers look fine, nobody may notice the issue until a very late and bad day.
“I don’t need SQL, I use Python.”
They told you to use Python, right? Maybe it is not the first thing to do in the data field, because the underlying concepts for manipulating data are the same ones as those you face in SQL, which is a more specific, dedicated language. Python is a general-purpose language that manipulates data using libraries like Pandas. These libraries create abstractions and provide operations similar to SQL: a merge is a JOIN, a groupby is a GROUP BY, and you can make the same mistakes as in SQL when you join tables without checking their cardinality in advance. Oftentimes, the numbers you manipulate in a notebook come from a relational database where somebody wrote a query, and when the number is wrong, you face people who extracted or manipulated your data source using SQL queries. At that point, talking in SQL terms simplifies the process for everyone involved.
There is a tax to pay later. With the excuse that “the AI does it anyway”, the cost behind it is more hidden, but it is still there. On your laptop, it doesn’t matter. The problem is that at scale, things are different. Making demos is one thing, and using things in production is another. I have seen thousands of demos. Turning a demo into something you use every day is a whole other story. And I cannot see someone who comes from Excel using something that magically produces a number for him without being able to validate it.
Being exposed to things you don’t need today helps you because it can open your mind to other things. I read a book years ago. How much of it did I need then? 15%. Today, I save time because when I read it, ChatGPT didn’t exist. You don’t want to do it; you are lazy, fantastic, but then don’t complain if they chose that other person instead of you. You can take shortcuts, but in the end, don’t complain if it doesn’t work.
When I say you must know SQL, it applies to both the person with 30 years of Excel experience who is now looking at this world and the person who just graduated. Both receive the same message, and it applies to both in the same way.
In a few years, the tools will keep changing, and AI will keep improving the quality of the queries it writes. Fewer people will write Python or SQL. However, those who skipped SQL in their education will still need external help to validate a number when they cannot understand how it was created.
No SQL, no analyst.