Do you really NEED business analysts, project methodology, etc.?
I work for a large government agency with a large IT department. I, however, am in another department. I've successfully self-trained to the point I am developing complex queries across different databases -- SQL Server, mainly, with some Oracle and legacy systems. I'm using Analysis Services, uploading data-driven subscriptions to Reporting Services, and doing other IT-type things for our department (yes, I do document my code and try to use Best Practices in all of these areas. I read a LOT to try to avoid awful mistakes.) When the IT department wouldn't give me a database on the network, I uploaded SQL Server Express to a machine under my desk and used it for Enterprise-level reporting. Once the users began to champion the reports I created, the IT department finally gave me a server on their network (with automated backups to the network and all that.) Now I use it as something of a data warehouse (data mart?) gathering information into usable tables from across a half-dozen other OLTP systems. It's been a great tool for reporting. Our IT department has a slew of Business Analysts and Project Managers. These positions "provide work flow analysis and documentation," "work within the Project Management Methodology framework," etc. What this means is that it's difficult to get much done at that level in the way of reporting and development. My department managers understand the problem, so I am their go-to guy when it comes to data needs. In addition, the IT department has to document their time in quarter-hour increments, and anything they do must be approved by others before they can work on it. It takes a long, long time to get anything done, unless I call someone with whom I have a good relationship and they do it outside of the bureaucracy. Every couple of years they'll decide that all reporting should be done by central IT. I go to the meeting, pull out a big binder of our reports, and begin explaining them. The new project manager for that meeting (it's always a new one) is shocked -- shocked! -- by the amount of complex reporting done at our department level. They say they'll get back to me, then I'm left alone for a couple more years. If one of my users wants a change to a report or legacy interface application, I can get it done within days, often within hours. I'll bring them a mock-up for approval, make the changes in a Dev enviroment, then run my own QA. I've been here for over a decade and have thorough knowledge of the business needs, so interpreting their requests is typically not problematic. My users are very happy, as is my department manager. There's no heavy layer of IT bureaucracy, yet I get things done. By myself. Is this situation normal within large organizations? I'd appreciate any insight.
Well, your title and description ask very different questions, so permit me to address both. The situation you describe is an extreme version, and no it is not normal for it to be that extreme. It is however perfectly normal for beauracracy to create slow downs. Sometimes this is a valid trade off because for all of the deragotory treatment a ***well run*** beauracracy can provide substantial value in a large organization. The beauracracy can create frustrating slow downs, but it can also prevent duplication of effort, help ensure smooth functioning, and amongst other things help to ensure that the organization as a whole can survive the loss of any one member. As a response to a beauracracy that is not well run (or due to a simple lack of official resources) it is quite common for individual users to write their own small applications which then may be picked up within a department or a large organization and become integral. It isoften quite common as those applications grow that they be transferred from the original creator to a more traditional development team. Depending on the details, this can be good or bad. If it means transferring it from a creator who is a hobbyist programmer at best to a well run development team that can optimize and expand the project and add functionality the original creator did not have the capability of doing, this can be a good thing. Especially if it means the original creator is now more free to focus on his primary job. If it means taking it out the hands of a skilled creator primarily to maintain "fiefdoms" within the organizations, that is less good. So, the basic situations you described are fairly common, but the way you have described it is an extreme version of it. Now, as to the title of the post, a ***good*** business analyst can bring a lot of value to the team. A good business analyst does not just create charts and reports, but will provide the context, interpretation, and *analysis* to the underlying business situation and this can help the decision makers make well thought out rational decisions. Whether or not one is needed depends on the type of project and size of the orginization, but they are certainly needed under certain circumstances. Even just a report writer can have substantial value. At a previous job I was the DBA with a small team of SQL developers working for me in a small but fast growing company. I handled creating most of the reports for the company. As the company grew, management decided, with my full support, to hire first one and then two full time report writers. The jr. report writer did nothing but develop report based on specifications given to her by others. The Sr. Report writer did some of the analysis I spoke of earlier, but that was really mostly laying the groundwork before handing the data off to specialists. Having them on the team meant I had more time to focus on the administration of the growing number of servers and work with our software developers while our specialists could get their reports fast (and they came up with new requests for reports regularly).
I've worked for multiple large organizations and no, I'd say that's not normal. At least, not to that extreme. Yes, every single one of them had a beauracracy and it did seem like certain projects and certain types of development ran very slowly, but... You also began to see how economies of scale could be achieved when teams of developers and teams of admins and teams of DBAs were able to cross-pollinate knowledge and build areas of discipline and skill. Then, you see larger projects done in the blink of an eye. You see innovation occurring and rewarded. Are individual projects and individuals sometimes given short-shrift in this kind of environment? Yes they are, but the question that gets asked, how much are they contributing to the bottom line. I remember a project we had that drove us all nuts it was so poorly run and didn't seem to be able to follow any of our standards and subsequently it had a lot of problems in production. We frequently referred to it as the clown car of development. And that project was responsible for managing 50 million dollars worth of company investements... But then it's for a multi-billion dollar company... and suddenly, you see why the clown car wasn't given all the attention that other projects were. Was it wasteful and inneficient? Yes. But the 10-20 million in savings we could have achieved there would have come at a greater cost because we would have pulled attention away from something worth 10 or 100 times as much. Just remember, a lot of the red tape and nastiness that is thrown up by DBAs are honest, sincere, attempts to protect the production systems, which is their number one job. Do they sometimes go overboard, oh yes. But if you remember what they're supposed to do you can work around them and with them to get done what you need to get done too.
As @TimothyAWiseman and @Grant Fritchey have stated, this is not really the norm. We have all experienced a lesser version of what you are going through. I would also like to point out that you sir are probably outside the norm as well. If I had a customer that had your knowledge was doing the type of work you are doing, you would have been drafted into the database group a long time ago. I have experienced situations very similar to what Grant has described. Situations where a project or process needed to be created and put it but frankly just didn't have the priority to get done. Individuals would proceed to tackle it on their own and put together a process that didn't quite fit into our standards. In many cases they are necessary evils. At some point the process ends up in the mainstream IT area to have to support and enhance and we love to butcher the crap out of it, even though it saved the company tons of money in process efficiencies. As for your original question do we need business analyst and process methodologies, absolutely. The end result with those two items has a greater chance of being completed on time, well documented, and put together with departments working together. I would much rather be on that project than flying by the seat of my pants in some covert operation.
I'd just like to add to all this, I came from a very small company where myself and my colleague acted as Project Manager, Business Analyst, Systems Analyst, Developer etc etc and I have to admit it was a complete nightmare. Now I work for a company where we have a Business Analyst, Project Manager, Developer and Tester that run the projects and it is so nice to have someone else take the responsibility for the various aspects. I do agree with all the others that the example given here is massive overkill, but in my limited experience having the shared knowledge and shared workload can be nothing but a benefit.
I spent seven years working at a company that had aspirations of being like the company you work at, but never bothered hiring anyone. That was an interesting time in my life. But hey, seems like you're doing OK being the go-to guy. Personally, I like being the go-to guy. The phrases "provide work flow analysis and documentation" and "work within the Project Management Methodology framework" literally make me want to poke rusty nails up my nose. For me, software is a very creative thing - and it sounds to me like that's the case with you too. However, at this point it probably seems like I'm answering at odds to everyone else. I'm not - I'm saying that those roles can fulfil useful functions, and can really add value (letting you concentrate on doing work instead of doing semi-project-management). They can also be used as a 'barrier' and I think that's the case in your situation. Down-side? Frustration and slow moving teams. Up-side? You get to be the go-to guy. Go superman!
All the other answers here has covered a lot of what I would say and it isnt uncommon. As DBAs we should try to bring people doing their own thing, such as yourself, into the routines of the IT department. Not to inflict the red-tape that you have described (which does sound like an edge case) but to make sure that the business isn't put at any risk. This is slightly off topic for your question but I feel it's important. You havent mentioned any of this specifically so I may be going off unnecessarily but ... Do you have backups on your system? SQL Express doesnt have and SQL Agent so you can schedule jobs for maintenance/backups data transfers etc If not then what happens if(when) your machine crashes? Your machine (presumably) isnt in a secure location such as your IT server room/racks - what happens if your office suffers a break in/fire/flood. Is your machine encrypted? Running from a UPS? Do you even have a RAID array rather than a single disk for some redundancy? What happens if you are away from the office - illness/holiday/new job - is there someone who can take over and continue to be the Go-To guy/gal? Are you legally allowed to have the copies of the data that you have? Do the business managers for that data know that it is on your computer? Do you get included in data audits when the other systems are scrutinised? If I was a business manager in your office I would want to cover all of this - fast. It's great to have someone like you in the team. Someone who can get the solution to a problem in a timely manner. Currently however I am concerned you are not in a team, you are an individual. If I was you I would take all of these points to the IT department and work hard on how to cover them all to their satisfaction. Currently they seem to be apprehensive of your work and are quietly ignoring it. You need to be working together to have their safety nets and security but your ability to fulfil a business need. Doing this you will concrete your value to the business and prove that you are thinking beyond the latest report request and into the future of the company. I hope this doesnt sound like I am picking on you, you have seen a need and taken positive steps to fulfil it when you have seen that no-one else could. No doubt your company is better off for that. I simply see that this could easily turn around and bite you if some mishap were to happen.
It depends... A good team will outperform an individual overall. We always followed the Magic Bus principal: what happens if you're hit by a bus? What happens to the users depending on what you supplied? We actually had a rule that not all members of a team could travel in the same vehicle. Good to hear you have the ability to take on these challenges and succeed and I feel for your frustrations. In addressing your post just remember that you have presented a unique scenario and your experiences aren't necessarily typical. Likewise, what potentially unnecessary resource demands have you made on other systems and the infrastructure to produce what you desire? You only "see" a slice of the big IT picture. An IT department should be examining it from a holistic point of view.