question

I have seen so many bad SQL statements written by developers and LINQ scares the hell out of me. I am afraid that with LINQ, even more developers will think that they can develop efficient database driven applications, and ignore the need for an experienced database developer, and in the end when the developers have left the building (consultants) the database performance will slowly degrades until the system is useless.

By then, when a DBA is involved trying to optimize the code, everything is build in C#, VB.NET or whatever, leaving the DBA without any options then add more hardware.

I think for small to medium sized databases,it is possible to write crappy code and the system still works (thanks to the hardware), but when it comes to large and huge databases it is not possible to solve the issues from crappy code with more hardware. The only solution is to modify the code.

Good question. SQL is surely a very poor platform for modern development needs. The fundamental language features of SQL DBMSs have improved very little in the last 30 years. Compare that to the major innovations in object-oriented languages over the same time period. Part of the problem is that the SQL model was fundamentally based on some flawed ideas and designed to work within the constraints of 1970s systems. The domination of the SQL DBMS market by three major corporations with vested interests in maintaining the status quo has also not helped.

I think the database management profession ought to be able to answer the question by saying "Yes, I hope so". We really ought to aspire to better things than SQL. I'm definitely not saying that LINQ is the right successor to SQL but at least it is one initiative that is showing some kind of way forward. The various products that go under the "NoSQL" banner are another example.

Unfortunately I think SQL database professionals have sometimes been a little too defensive about SQL and this has left them somewhat outside of the discussion rather than participating in it and putting forward alternatives. I think more data management professionals at the grassroots level need to recognise the need for new alternatives to SQL and contribute to the development of those future solutions. For instance, where are the relational-based alternatives to the SQL DBMS model?

As long as we have structured data, there's going to be a need for a language to query against it. That's not to say that SQL in it's current form is the way to go, but some type of language that is capable of dealing with pulling data out of relations within structured data is needed, and so far, none of the alternatives is more workable than SQL, in fact, most are less workable. Both literally and figuratively, in terms of an alternative to SQL, I'm from Missouri... Show me.

Until I see a fully functional alternative, I'm going to stick with SQL. That means that all the problematic ORM tools are going to remain, to a small degree, a thorn in my side. They don't deal well with SQL or relationships. They are, in my opinion, an attempt to ignore SQL, not replace it or deal with it. That, unfortunately, is the worst possible approach. Better to deal with it badly than try to pretend it doesn't exist and then generate all the horrors that I've seen & heard about.

However, get me a better method of querying the database, and I'm all for it. There's no reason to be married to a language. Based on the last 60 years of computing history and my own 20 years within it, we will get a different language eventually. Languages, OS's and platforms come & go. Nothing is carved in stone. But, at this time, there's nothing viable as an alternative that I can see.

My first response is "no, SQL works well"

However the note by dportas is interesting. Should we be looking at a way to possibly evolve to a new language that works better? We have seen "C" evolve to C++, C#, and now F# in some ways. Python is a very interesting environment to me as well, but in spite of all the work done on languages, there's still a decent amount of C around, and I wonder how much C++ code is actually C compiled in a new IDE.

From what I've seen in terms of how we work with data, SQL works pretty well, but it has a decent learning curve and requires someone to fundamentally be able to understand how SETs interact. That feels like the pointer issues of my early career where lots of people just could not understand a pointer.

If I had to say that SQL fails in one place, and I've seen this with the intellisense challenges, is that it doesn't fundamentally order the commands correctly. In my mind we ought to decide what tables we are using first, then get columns and relationships. So something like

FROM Customers c
  Inner Join Orders o
     on c.custid = o.custid
SELECT c.customername, o.orderdate, o.ordernumber
 where o.orderdate > '1/1/10'

I think that we possibly could also find some other ways to create set interactions, or perhaps even have FKs automatically joined with a keyword, could build cleaner code. The whole change to using a CTE in SQL, with the code at the top, to me, means it's fundamentally harder to read. That was, in some sense, a step backward by bolting something on rather than fundamentally re-working the language.

I think LINQ wasn't a bad idea, but the way it translates into SQL is what I find problematic.

I think the point made by Scot Hauder in his comment is worth highlighting (although maybe his intended meaning wasn't the conclusion that I draw from it):

"what may seem like modest change in logic or design of a query can literally alter the execution time from seconds to tens of minutes, or even hours"

This is true and it perfectly illustrates what a failure SQL is as a database language. The point of Physical Database Independence is precisely that the logical should not determine physical. The SQL language's redundancy, over-complexity and lack of orthogonality mean that it utterly fails to do that.

There are a lot of thought provoking comments here. I will avoid rehashing the ones that have been repeated over and over, but one thing no one has mentioned yet that I saw is how language specific the ORMs are. If you want to use Linq you must use a .Net language. If you want to use SqlAlchemy you must use Python, etc.

This will make it hard for any one ORM to gain dominance. The core of SQL is implemented in numerous databases though. They each have differences and quirks, but going from Oracle to SQL Server is like the difference between American English and British English, nontrivial but easy for someone who has mastered one to figure out the other very quickly.

I'm a non-SQL expert but have been using it for 20+ years as a developer/admin. Every time I need to do something more complex than a few JOINs and subqueries it seems that I need to dig into the documentation and iteratively test various syntax forms before it starts to work. For those of you that live and breathe SQL, it probably flies off your finger-tips without errors and without requiring much conscious thought. When I have time to frame the query (or other CRUD) in a human-understandable fashion, most of my coworkers can deduce what I am trying to achieve and think that should be easily achievable. Yet it always takes several attempts to phrase it in SQL that ~may~ bring back some of the desired results. Perhaps playing multiple roles in most of the companies that I've worked in (DBA, web/app programmer, manager/marketer) gets in my way. If I were only an SQL programmer all of these fine points about SQL and LINQ and optimizations would be just what I would be looking for. My current job involves a DB that has evolved over 15 years (Access -> MSSQL). There has been a strong attempt to capture most of the business rules (and some UI) in a plethora of stored procedures and user-defined-functions. These procedural elements can't be organized in any logical/hierarchical sense (not supported in MSSQL) so we end up with 400+ objects that only their developers can understand. There is no class encapsulation of functionality that I know of other than invoking yet-another function that is in the same top-level hierarchy. OK, so too many peeves before I get to my main one: Our DB dataset is only around 2GB. We could suck this whole thing into a set of linked objects in memory and process them using almost any language other than SQL. Yet, there doesn't seem to be a good query language alternative that supports all of the conditional, joining, and selection logic of SQL. Any suggestions?

Idea triggered by reading the "non-SQL expert's" frustration with many non-grouped functions: It would be nice if SQL Server could **group objects** (functions, procedures, tables, views) into **user-defined groups/subgroups** *with the sole purpose of order* and thereby **saving time** getting an overview of a database. This will NOT change SQL and should be relatively easy to implement. PS! This should be a comment (and is going away from main question), but I cannot make comments. I think I need "50 karma points" to be allowed to comment.

I agree with many here that SQL is not on its way out. SQL will be around for a long time in one fashion or another.