Applied Visual Studio Team System : SSIS

Attending the PASS Summit

andy — Sat, 22 Sep 2007 08:33:00 GMT

Steve Jones makes some good points in his blog post Training. I find it difficult to believe the short-sightedness of some organizations when it comes to training events like the PASS Summit.

This year's Summit - like all previous years to date - had enough top notch presentations and labs to make it worth the cost of admission, travel and expenses, and the cost of allowing a database professional to leave work for three days combined. More than enough.

Like Steve, I don't get it.

Also like Steve, I bet we'll see these DBAs at the 2008 PASS Summit in Seattle - and working for another company.

I wonder if those responsible for denying database professionals opportunities for training factor in the cost of hiring and training a new DBA every six to eighteen months?

:{> Andy

Technorati Tags: EMPs Database Professionals PASS Training Changing Jobs

At the 2007 PASS Summit!

andy — Wed, 19 Sep 2007 05:19:00 GMT

I'm at the 2007 PASS Summit!

I was wandering around the Colorado Convention Center earlier today in shorts, unshaven, tired... looking like I'd been rode hard and put up wet. But I caught the last half of Gert Draper's excellent presentation on Team Edition for Database Professionals! Gert is the man.

I'm presenting on the same topic tomorrow (yeah, I know - great move there, Andy...) and then on SSIS Development practices Thursday. This promises to be the geekiest week I've had in a long time!

:{> Andy

Technorati Tags: PASS Summit 2007 Denver Team Edition for Database Professionals

Getting Ready For The PASS Summit!

andy — Fri, 07 Sep 2007 21:31:00 GMT

The PASS Summit is less than two weeks away!

I'm getting ready for my presentations. I need a couple laptops to host virtual servers for the demos, so I bought some new gear to take with me.

Check out my Network-In-A-Bag!

It's a power strip, a couple CAT6 cables, power supply, and a NetGear 1G 5-port switch - all in a 1 gallon Ziploc bag.

:{> Andy

Technorati Tags: PASS Summit 2007 Networking

Iteration = Maturity

andy — Tue, 31 Jul 2007 14:01:00 GMT

Introduction

I was recently reminded that iteration matures software.

The History of Andy, Part 1

Like many DBAs, I was a software developer in another life. I built web applications - working my way up from HTML through DHTML and finally to ASP - and could regale (and bore) you young whipper-snappers with war-stories of how things were "back in my day". [/DanaCarvey]

But I won't.

The Times They Are a-Changin'

I'll share instead something I've witnessed many times since starting with software in 1975 - and something you probably already know: stuff changes.

And thank goodness stuff changes!

I recently ordered 1G of RAM from an online retailer. It should arrive before my next son (but that's not a given as Riley refuses to provide a tracking number - the doctors will induce Christy into labor Friday if he hasn't been born by then - but I digress...). I remember my neighbor John, who introduced me to computers, purchased a 256-byte RAM chip in the mid-1970s for about what I paid for the 1G. That's 256 bytes of RAM - not a typo. As I recall it was either a 14- or 16-pin IC.

Things have changed since then. Improvements in technology, brought about by building and improving upon existing knowledge, have brought us to a day when I can purchase 1,073,741,824 bytes for roughly the previous price of 256. I don't know how you feel about that. I think it's a good thing.

The idea of "building and improving upon existing knowledge" defines iterative development. Although the idea is relatively new to the software development field, it serves as the basis for engineering disciplines. Engineers iterate - build and improve upon existing knowledge - and we get more powerful hardware for the same amount of money. What's not to like?

Iteration - it's not just a good idea...

Iterative software development builds and improves upon existing knowledge within a specific domain. Most domains are defined by an application (wholly or in part), enterprise knowledge (again, wholly or in part), or - most likely - some combination of the two. For example, let's say you work for a large corporation as a software developer. Your domain could be the corporate website. In which case you possess knowledge about the business of the corporation and web development. You mix these together to do your job. In this case, you will probably pick up marketing savvy and current trends along with the latest AJAX techniques.

As you make successive passes (iterations) through the website design interacting with marketing, your domain knowledge is built and improves. As your domain knowledge increases, the website will become more valuable to the corporation - as will you.

Iteration adds value.

Got Iteration?

The same can be said for database development.

Perhaps you've experienced this in your own database development efforts: you receive a request for a database design to meet some desired functionality. Or you're handed a design and asked to optimize it. Or maybe even you had an idea to capture data - performance metrics or something similar - and you're designing a database solution to accomplish this.

You get into the development a few hours or a few days and realize a little tweak here or there would improve performance, or readibility, or better adapt the design to your intentions. So you make the tweak and continue.

This improvement leads you to re-examine other portions of the design and you make more tweaks. Maybe your last change broke things. Maybe you see an opportunity to add a parameter to a stored procedure and combine the business logic of three stored procedures into one.

A "Growing" Solution

Pretty soon, you have iterated enough to feel comfortable promoting, integrating, or even releasing the results - letting the effort move to the next step.

Depending on the nature of your efforts, it may not end there. If your database development is the back end of a larger application - say, the corporate website, for example - there will likely be requests for changes over time as the site grows (scales) in complexity and size.

When the requests come in you are not likely to start over. You will most likely build and improve upon your existing knowledge. You will most likely iterate.

Scaling forces iteration.

Voilà

This is how solutions mature - be they applications, databases, or both - regardless of who writes them or how many are involved in the development effort. It doesn't matter if the development team is one lady in a cubicle in the European Union or a development team of thousands at Microsoft.

Iteration matures software.

:{> Andy

SSIS for DBAs

andy — Mon, 30 Jul 2007 19:43:00 GMT

Solid Quality Mentors is now offering a course entitled SQL Server 2005 Integration Services for Database Professionals.

I was honored to work with Erik Veerman to develop this SSIS for DBAs course! Our goal is to provide training in SSIS's non-ETL capabilities - stuff that is useful for Database Administrators.

Although it is an amazing ETL engine, SSIS is more than an enterprise ETL platform - it has lots of capabilities built especially to assist DBAs in the Production environment. In the course, we introduce the SSIS development environment and SSIS to folks who may have never opened Business Intelligence Development Studio.

Next, we cover the basics of SSIS - walking through capabailities of the Control Flow, Data Flow, Event Handlers, Variables, Properties, and Expressions.

We then get busy with Transfer Tasks, Import and Export Wizards, and Maintenance Plan development. Our next section focuses on using SSIS to perform more advanced DBA tasks - building SSIS packages that automate FTP, flat file data extraction and loading, and file archiving operations. We cover optimization, data cleansing, text mining, and binary data. We also dive into WMI, File watching, and Notifications - there's even an optional section on migrating DTS!

We then address SSIS administration, deployment, security, package configurations, logging, modular package design and team development. We cover restartability, snapshots, and transactions, and conclude with a section dedicated to troubleshooting.

Material is presented, then demonstrated. Students then perform lab exercises to reinforce the presented and deomnstrated concepts. It's a great way to learn - especially in a rich visual environment like Business Intelligence Development Studio.

As if all that wasn't enough, the course is led by Solid Quality Mentors! These are folks who have been there and done that - database professionals that are published, MVPs, or both!

If you're interested, contact Jeanne Reeves at Solid Quality Mentors.

:{> Andy

Technorati Tags: SSIS DBA SSIS for DBAs Database Professionals Solid Quality

Meltdown!

andy — Wed, 04 Jul 2007 19:46:00 GMT

A couple days ago the Vista Ultimate instance on my laptop when all Klingon on me: it was "a good day to die."

I'll never know why for sure. Indications point to COM+ and VMM giving up the ghost. They were good systems, may they rest in peace.

I tried to revive the old OS. It would run in Safe Mode and even Safe Mode with Networking, but that's just not the same as having all the functionality I know and love.

I've spent the last couple days (and nights) rebuilding a second instance of Vista Ultimate on the same machine. I'm about half done at this point. Today is SQL Server instances and Visual Studio Team System day.

It's not just the installs, there's the service packs and updates after the installations. Lots of installing.

But it's also an opportunity to rebuild the machine with a different configuration.

Back to installing...

:{> Andy

Technorati Tags: Vista Re-install

Raleigh Code Camp - tomorrow!

andy — Sat, 23 Jun 2007 05:14:00 GMT

I'm looking forward to speaking at the Raleigh Code Camp tomorrow!

I'll be presenting on Team Edition for Database Professionals and Incremental Loads with SSIS. It going to be fun!

If you're atending tomorrow and read this blog, please introduce yourself!

:{> Andy

Technorati Tags:

Back in the Saddle...

andy — Sun, 10 Jun 2007 12:01:00 GMT

Monday, after a five-month hiatus, I return to SSIS training.

I really enjoy training whether I'm attending or leading it. It killed me last week to miss both Tech Ed and the Richmond .Net Users Group June meeting, but a client needed me on-site and I try to never leave a client hanging.

I roll out around 6:00 AM tomorrow to head for the airport, and it's 1:40 AM as I type this - another day when it's good to not need sleep! :)

To the good folks at New Horizons in Edina, MN: I'll see you Monday!

:{> Andy

Technorati Tags: SSIS training Minneapolis MN

SQL Server 2008 CTP Available!

andy — Wed, 06 Jun 2007 20:29:00 GMT

The June CTP of SQL Server 2008 (the database server formerly known as Katmai) at Microsoft Connect!

Registration is required. You must also complete a three-question survey.

:{> Andy

Technorati Tags: SQL Server 2008 Katmai CTP

Managing The Thing You Cannot Touch

andy — Sun, 27 May 2007 07:01:00 GMT

Yesterday I wrote about The Thing You Cannot Touch. Today I'm going to tell you some ways to manage the situation.

First, try to determine why You Cannot Touch The Thing. This is invaluable information in charting the waters ahead - especially if you're consulting.

Second, accept the fact that there's better than a 90% chance that you will not, in fact, be allowed to Touch The Thing. In my experience, three things must be true for you to overcome the business friction imposed by The Thing:

You have to try everything else first.
Everything else must fail to sufficiently address the issue.
The source of the issue must be mission-critical.

Regardless, your best knee-jerk reaction is acceptance. This is tough for a professional. In your heart of hearts you know what it takes to solve the real issue. And yet, you've been told You Cannot Touch It.

The good news? There's also a better than 90% chance you can find a way to solve the issue - or at least alleviate the client's pain - without Touching The Thing.

Modern enterprise applications are comprised of lots of moving parts. The Thing is probably not the sole source of pain. Addressing other bottlenecks may do the trick - at least for now.

And, if you're the person they called last time they had an issue and you solved it (and weren't "difficult" to work with), you'll likely get the call next time.

How cool is that?

:{> Andy

Technorati Tags: Consulting Software Development Satisfying The Customer Leveraging New Business

The Thing You Cannot Touch

andy — Sat, 26 May 2007 07:01:00 GMT

I have this theory about consulting. I call it The Thing You Cannot Touch. Since a few friends have found it amusing I thought I'd share. It goes like this:

A potential client contacts your firm. A conference call is arranged to discuss the issue. During the call, the issue is defined. Resolution theories and attempts to date are shared, along with their results. The current status is explained - along with

The Thing You Cannot Touch.

Sometimes an attempt at justification accompanies the announcement: "We know it can't possibly be _______ so we're not going to waste any time looking at it."

Other times, it's just put out there for what it is: "You can't touch _______."

My experience has shown the heart of the issue almost always lies with The Thing You Cannot Touch. It needs to be fixed but someone, somewhere, for some reason does not believe it to be so - and so it Cannot Be Touched.

Sometimes it's political - It's someone's "baby". They built this application just ten short years ago - worked nights and weekends and toiled and sweated and bled to make it work - and rode it all the way to CIO, after all. Who are you, lowly consultant, to tell them VB 6 code should be re-written in this new fad known as .Net? Doesn't Vista support VB 6 until the mid-20-teens?

Sometimes the decision-maker doesn't understand the differences in the technologies.

Sometimes it's a purely market-driven business decision - and the decision-maker is right and justified in choosing to keep hands off The Thing. It's not all about technology folks... it's sometimes about what I like to describe as the (little "s") software (big "B") Business.

If you find yourself on a consulting conference call and The Thing You Cannot Touch comes up, pay attention. Tomorrow I tell you how to Manage The Thing You Cannot Touch.

:{> Andy

Technorati Tags: Consulting Software Development Thing You Cannot Touch Old Code Outdated Code VB 6

SSIS Design Pattern - Incremental Loads Post Now Live at ABI

andy — Thu, 24 May 2007 07:01:00 GMT

The latest in my SSIS Design Patterns series - SSIS Design Pattern - Incremental Loads - is now live at Applied Business Intelligence!

:{> Andy

Technorati Tags: SSIS Design Patterns Incremental Load Business Intelligence ETL BI

Updating the Virtual Team Foundation Server document

andy — Thu, 10 May 2007 13:00:00 GMT

Talk about overdue... I'm finally updating the Virtual Team Foundation Server documentation at VSTeamSystemCentral.com.

There are 163 pages out there now that demonstrate the procedure I use to build a vTFS but they were written for the December 2005 CTP of TFS. A couple things changed - not much mind you: I think the biggest change is less permissions (not a member of local administrators) for the TFSReports and TFSService accounts. This means the document is ok as far as functionality is concerned, but it's always best to follow the principle of least privilege.

I'm finishing up some writing and a project in the next couple weeks. Adding some content to my blogs and VSTeamSystemCentral.com is one priority. I have about 35 blog posts in my \Andy\Blogs\Primordial folder just waiting to be posted - most of them at Applied Business Intelligence in a series I'm calling SSIS Design Patterns.

I really enjoy having too much work to do - I like it a lot more than the alternative. But I'm also looking forward to doing more work around the house as Wee's birth approaches.

:{> Andy

Technorati Tags: Team System Team Foundation Server SSIS Design Patterns Business Intelligence Wee Leonard

SSIS Design Patterns Series

andy — Tue, 03 Apr 2007 07:58:00 GMT

On Applied Business Intelligence I've started a new series called SSIS Design Patterns.

The first in this series is now posted: Dynamic SQL.

:{> Andy

Technorati Tags: SSIS Dynamic SQL Design Patterns

A Green Box

andy — Thu, 29 Mar 2007 03:04:00 GMT

It's been way too long since I posted a blog post - my apologies! I've started a bunch of posts (mainly so I wouldn't forget the idea on the top of my noggin), but nothing complete enough to turn into the wild.

I've been swamped on a consulting gig. It's a good gig - just intense. Since I started doing lots of SSIS training for Solid Quality, I had this idea it would be good to do a few weeks of consulting every now and then - just to stay sharp.

After battling a particularly "interesting" requirement, I finally got this:

Most of you reading this are developers. You know why I'm posting - it's the 11th hour of a 12-hour day. And yesterday was a 12-hour day too. But tomorrow is a really long day, but it's travel-home day and Stevie Ray's P.E. and Show-and-Tell day. His teacher tells us that for the past six weeks, Stevie's Show-and-Tell item has been the same: "Dada's coming home tonight!" Heart-warming and -breaking, all at the same time.

So this is a celebration. Houston, we have a green box.

:{> Andy