18 Questions to ask yourself when choosing continuous integration and build automation tools

My blog has moved. You can view this post at the following address: http://www.osherove.com/blog/2010/1/21/18-questions-to-ask-yourself-when-choosing-continuous-integr.html
Published Thursday, January 21, 2010 4:57 PM by RoyOsherove
Filed under: ,

Comments

Thursday, January 21, 2010 10:58 PM by EricMinick

# re: 18 Questions to ask yourself when choosing continuous integration and build automation tools

The other question I see with CI tools (disclaimer: I work for a vendor) is how far do you want to go with CI? A major purpose of CI tooling is testing your builds after a code commit and letting everyone know how OK things are.

Are you just compiling? That's a great first test. Running unit tests? Even better. Deploying to a test server and running some automated functional tests? Better yet. Continuous Deployment to production IMVU style? Moderately crazy, but very impressive.

How far you want to push the limit will inform what kinds of capabilities you need. Will a single machine work or do you need a tool with server-agent capabilities? Is this just developer feedback or are we managing the hand-off to test or operations teams?

Thursday, January 21, 2010 11:46 PM by Natural Cause

# re: 18 Questions to ask yourself when choosing continuous integration and build automation tools

Haha, I like the Bonus question. I'm not even sure if i'm after a CI or BA tool.

What I want it something that will automatically automatically kick off a get latest from my SVN Repository, Build the project, Run the unit tests, and Deploy the site to staging.

But not sure what I'm after other than i hope it's freee cos its for personal use.

Friday, January 22, 2010 1:32 AM by tmb

# re: 18 Questions to ask yourself when choosing continuous integration and build automation tools

Am I using someone else's checklist or do I make a decision based on my own list of criteria?

Friday, January 22, 2010 2:36 AM by My thoughts, shares ... with .Net and Microsoft

# 18 questions when choosing Continuous Integration and Build Automation Tools

Friday, January 22, 2010 3:37 AM by The Morning Brew - Chris Alcock » The Morning Brew #523

# The Morning Brew - Chris Alcock » The Morning Brew #523

Pingback from  The Morning Brew - Chris Alcock  » The Morning Brew #523

Friday, January 22, 2010 4:48 AM by 18 Continuous Integration Questions

# 18 Continuous Integration Questions

Pingback from  18 Continuous Integration Questions

Friday, January 22, 2010 4:52 AM by Julian Simpson

# re: 18 Questions to ask yourself when choosing continuous integration and build automation tools

Roy,

Nice work  - I liked this post and it's predecessor.  I blogged this and asked some some more questions at <a href="www.build-doctor.com/.../">The Build Doctor.</a>

Friday, January 22, 2010 5:35 AM by Jesse Gibbs - Atlassian

# re: 18 Questions to ask yourself when choosing continuous integration and build automation tools

Disclaimer:  I work for Atlassian, a vendor of CI tools.

A few more things to consider include that really help determine how easily a tool can be adopted by the entire team

* Does the tool let me use my existing automated tests (Unit and functional/UI) without having to modify them?  Does it display the results in a sensible and useful way?

* Does the tool notify me of important events (failed builds, hung builds, etc.) using the notification methods I prefer (IM, email, RSS, desktop/ IDE pop-up)?

* Does the tool integrate in a useful way with my bug tracker, IDE and other parts of the existing tool stack?

* How will I scale the CI system if the team adopts CI and needs additional computing resources?

Friday, January 22, 2010 10:11 AM by John Sonmez

# re: 18 Questions to ask yourself when choosing continuous integration and build automation tools

Good post, I have run into all the needs you describe as questions above in putting together my CI server.  I wrote a post about best practices for CI here: simpleprogrammer.com/.../continuous-integration-best-practices

I am using Hudson on a Java project and it is quite good, although not perfect.  Configuration is really easy.

# 18 Questions to ask yourself when choosing continuous integration and build automation tools &#8211; ISerializable &#8211; Roy Osherove&#039;s Blog

Pingback from  18 Questions to ask yourself when choosing continuous integration and build automation tools &#8211; ISerializable &#8211; Roy Osherove&#039;s Blog

Monday, January 25, 2010 11:29 AM by Scott Stanton - Electric Cloud

# re: 18 Questions to ask yourself when choosing continuous integration and build automation tools

Disclaimer: I work for Electric Cloud, a vendor of BA and CI tools.

This is a great list of questions so far!  When you want to scale up past an individual team there are a few extra questions to ask:

* How easy is maintenance on the system?  Can I upgrade the system without interrupting long running builds?

* Can I reuse existing corporate infrastructure? (e.g. Oracle, Active Directory)

* What is the strategy for supporting multiple groups in the company?  Can they share resources?  

* What support is there for data security or access control?