Thursday, February 03, 2011

Comparing Jenkins vs TeamCity - Part 2

This is a second installment in a series of posts that tries to compare two continuous integration and build automation server products. In my previous post I briefly introduced the contenders open source developed Jenkins and a commercial offering from JetBrains called TeamCity and compared their respective installation experience.

This time I'll try to cover the post-installation configuration steps I need to make before I can start putting some builds into them.

Configuration

No server software is usually useful out of the box in and by itself. You almost always need to tweak and configure it before you can really reap the benefits. Continuous integration is just another type of server and it needs to be configured before it becomes really useful.

TeamCity

After you've installed your copy of TeamCity and opened up it's web interface for the first time, it greets you with a nice little license agreement, you need to agree to in order to be able to continue.

This is just a first page of the initial set-up wizard:
  1. License agreement: Check "agree" and press Continue
  2. Fill in details for admin user account and press "Create account" button
Initial Overview screen
After this really simple initial set-up wizard you're basically configured with a simple authentication scheme, have created one administrative account and are ready to go.

However, not everything is configured, as is also suggested by the projects initial project overview page.


    Email and IM notifications.

    One of the nicest things about CI services is that they can notify you proactively if you need to know of some important events like builds or test failing or some such.

    Email notification settings
    In Out of the box configuration you can set up your e-mail and Jabber notifications. Just follow the link that says "configure email and Jabber settings" in the Overview page.
    It leads you to Administration > Server Configuration section.
    Then click on the "EMail Notifier" tab, fill in your e-mail SMTP gateway details, press "Test connection" to ... well, test the connection. and if it checks out, press "Save".

    If you use internal (or external) Jabber service for instant communication, you can also set it up by filling out a similar form under the "Jabber Notifier" tab.

    Database

    By default, the TC installation comes with embedded Hypersonic SQL DB (HSQLDB for short). It is a good database in it's own right and has it's strengths in the situations where you would need a fast and tightly integrated in-process DBMS embedded into your Java application, but running a production server on HSQLDB is ... well lets just leave it at "it's just not done all that often".

    JetBrains nicely offeres a link to migration instructions on TC's server configuration administration screen.

    Basically there are two ways to migrate.
    1. Wipe TC data location, specify new db configuration and essentially re-configure your instance
    2. Full data migration
    First choice suits you perfectly if you have just installed your first instance of TC and want to store the data in something else than HSQLDB.

    The second one is what you'll do if you have run for a while, have some projects and users configured, etc. In short, if the current DBMS you has some data, you do not want to lose and you need to migrate it to another database for whatever the reason.

    TeamCity supports HSQLDB, MySQL, Oracle, PostgreSQL, Microsoft SQL Server 2005, 2008 and Sybase; the migration is possible between any of these databases.

    Migration in general is a tedious process, involving backing up data, stopping and re-starting server(s) and generally doing lots of  manual labor. The authors of TC have done a fairly good job of explaining it all.

    Jenkins

    After you've installed Jenkins in the manner outlined in a previous article, you can open your browser, point it at the http://localhost:8080/ and you are presented with a Dashboard, where you can immediately start by adding new build projects.

    Out of the box configuration of Jenkins allows you to pull your sources from CVS, Subversion and local filesystem; build Ant or Maven driven java builds and execute shell/command line commands and scripts.

    So if all you need is a local build server instance in your machine, to build your java projects, you're pretty much done.

    Still there are some things missing from the default configuration that are sort of mandatory, if you are working in a team and building up your CI system in earnest.

    Basic security

    Out of the box installation of Jenkins has no security whatsoever. In essence it is a public free-for-all build server, where anyone has full access to alt the builds, full configuration and all of the operational details.

    It is probably just okay in a small company or department's internal closed network setting, or as a personal server, where everyone is fully trusted to do the right thing and not to mess up a production grade build automation. However as soon as your build server has to have public/internet availability you'll need to start turning down those knobs and that is where additional security measures come in handy.

     Setting up basic security isn't hard:
    1. From Dashboard: Manage Jenkins Configure System
    2. Check "Enable security"
    3. Select how your users will be authorized (e.g. LDAP or Jenkins's own database)
    4. Choose the level of security you'll feel comfortable with
    Hit "Save" at the end of the page (if that is all you needed to configure), and you are secured.

    I must however point out that the vanilla installation of Jenkins comes only with few user authentication options, but you can install additional methods from it's extensive set of authentication plug-ins

      Email notification

      As with TC you need to configure your e-mail gateway before it can start sending e-mail notifications.

      Like everything that has a configuration setting, Email notifications section is on the System Configuration page. Just open by navigating to Manage Jenkins Configure System, scroll to the bottom of the page to the "E-mail Notification" section, fill in your SMTP gateway details (if your SMTP server requires logon, do not forget to click on "Advanced" button and fill in those advanced e-mail settings)

      Test the  configuration and press "Save". Pretty much same as TC.

      Database

      From all that I've seen (not that I looked too deep), Jenkins does not use any SQL database for persisting it's state - using custom XML based persistence instead. This is good, because this means you can do ton of things with the data, like store it ins some VCS, clone settings into new Jenkins instance, back it up, script it up, etc.

      Conclusion

      First difference I noticed is that the first thing TC did was to create an admin account and thus secure your TC server installation.One might argue if it is a good thing or not either way, but the fact is that the very first entry on Jenkins best practices page recommends "Always secure Jenkins.".

      At the very least what I would like to see is a nice friendly reminder in a dashboard, that says something to the effect "This copy of Jenkins is completely unprotected". You could dismiss it if it bothers you, but it should be visible at least the first time you visit unsecured Dashboard of a freshly installed/updated Jenkins instance.

      A nice touch of the TeamCity was the "unsaved changes" notification at the bottom of the browser window when you had modified something, but had not hit the "Save" button yet. It served a double purpose of reminding you that you had unsaved changes on the form and offered a nice easy access to the "Save" action.

      Overall, here is where TC commercial backing shows - the overall out of the box experience is somewhat smoother and nicer than Jenkins's.

      Enterprises will love the fact that TC keeps it's data in an SQL database and the lineup of supported databases should satisfy any corporation, but the simplicity of persisting data in XML files is just barely short of genius - Developers (and dare I say, sysadmins) will love this.

      The set of minimal post-installation configuration steps will vary by use case for either, but both products make it rather painless process and all things considered I'd like to give this step a tie.

      What's next


      In my next post I will configure and build my first project. Stay tuned

      Comparing Jenkins vs TeamCity - Part 1

      As some of my closest friends already know, I decided to bite the bullet and switched jobs recently.

      As the new company I now work for is really small compared to the one I left, it also means that as a developer I am also involved in more than just programming.

      One of the decisions I had to make and implement was selecting and setting up a CI server for continuous building and deployment of our next product.

      Having had extensive exposure to CruiseControl, I immediately discarded it as I've developed a particular dislike for this product.

      The next alternative I had had any exposure to was Hudson (now renamed to Jenkins) and JetBrains's TeamCity, which I had heard a lot of nice things about. Adding to the mix that first project to hit our CI server would be .NET project, I needed a solution that would run on Windows and build .NET projects using MSBuild.

      I settled on Jenkins and TeamCity as the two major contenders.

      I would like to point out that this post is full of personal opinions, by no means impartial as comparisons go and should as such be taken with a fairly grain of salt. You might not like it or not agree with my conclusions, and that is entirely your tragedy. I just record my experiences here - YMMV.

      Introduction

      First a little introduction of the two products.

      TeamCity

      TeamCity (TC for short) is a commercial offering from JetBrains (the company who have brought to Java development community an excellent IDE called IntelliJ IDEA).
      JetBrains offeres it as a Professional and Enterprise editions. Professional edition is free and is limited to 20 user accounts and 20 build configurations. With the product you get to use up to 3 build agents and you can purchase more as you develop a need for these.

      20 users and 20 build configurations sounds plenty fine for our case, so this is no problem at the moment.

      Jenkins

      Jenkins (originally called Hudson, until Oracle botched it with the community and got renamed) is an open source CI server implementation that started out as a hobby project by Kohsuke Kawaguchi while he was working as Sun, but has now grown into one of the most popular CI server implementation in the Java community. It has a vibrant community of core and extension developers, rich set of plug-ins that let it do almost anything that a CI build server needs to do.

      As an OSS project, it is completely free. No strings attached, no limitations, no hidden agendas.

      Installing

      First things first. Although there are cloud offerings out there that offer CI server configurations, the requirement to build .NET projects, to really try both out, I need to install them side-by side.

      Installing TeamCity

      Here are the steps:
      1. Download the installer for free Professional version from JetBrains - 301 Mb
      2. Run the installer and follow the steps within
        Sit back and wait until TeamCity installer finishes setting itself up.
      3. Answer few more questions like operating port of the build server, set up build agent properties and some more stuff that seemed to have fairly sensible defaults.
      4. Open your browser and point it to http://localhost:80/ (or whatever port you chose to run your TeamCity server on)
      TC download was huge and it took about 30 minutes to download the thing.
      After installer finished running, the server and agent services were installed and running.
      Nothing more to do.

        Installing Jenkins

        1. Download the latest jenkins.war [1] - 35 Mb
        2. Run fom console: > java -jar jenkins.war
        3. Open your browser and point it to http://localhost:8080/
        All in all the whole process took me less than 30 minutes from the moment I started my download until I had Jenkins server up, running and ready for action.

        To be completely honest, these steps will only run Jenkins instance and as soon as you kill the process, or shut down your computer, Hudson will go down. To finish the installation of Jenkinbs you need to install it as a service on your host machine. The instructions for that vary by platform, but under Windows it is just as simple as following links from dashboard:
        1. From Dashboard: Manage Hudson Install as Windows Service
        2. Enter path to Jenkins service home (e.g. C:\Build\Jenkins)
        3. Press Install button
          Wait until Jenkins installs itself as a Window service
        4. Press "Yes" to stop currently running Jenkins instance and start the windows service.
        Alternatively, if you have a servlet container that supports Servlet 2.4/JSP 2.0, such as Glassfish v2, Tomcat 5 (or any later versions), then you can run them as services, and deploy jenkins.war as you would any other war file. Container specific documentation is available if you choose this route.
          [1] at the moment of this writing it was still called hudson.war, but has since been renamed to jenkins.war

          Conclusion

          Installing both was relatively easy process.
          • With TeamCity you get nice rouunded up installer package which handles all the gritty details of setting up app server, deploying applications and wrapping it up into a system service. Just download, run the installer, answer to some fairly simple questions and you'r ready to go.
          • With Jenkins the process was even simpler, albeit a bit technical, requiring you to go into the command prompt to run the server.
          I personally liked Hudson way a bit more as it felt really light-weight and to some extent much simpler - "just run the .war file" and you're done.

          Additionally - installing it as Windows service just blew me away - Like it? Take it - Click, click, clickedy-click and Done!

          Definitely, Jenkins feels much friendlier towards "just trying it out" and it can not be bad to the developer community - using Hudson, it is not entirely impossible to have each developer running their own Hudson CI instance on their own machine - it is just so simple to set up!

          What's more, since it is so easy to set up (and you don't have to commit to it by installing it as a service), it has a real advantage of being the CI of choice for every developer. Every one can run their own Jenkins instance on their computer. And that is good...

          What's next

          As I started to write this article, broke it up into some chapters and wrote few of them, I realized that this one has a fair chance of becoming a monster article, so I decided to brake it up by topics.

          This one established some background and covered installation of the server.

          In the next post of the series I'll be covering post-install setup of both products