First Swiss OpenStack user meeting (notes from Tim Bell)
Around 90 people attended which provided for a lively atmosphere with a good opportunity for audience interaction and participation (photos are here ). Discussions continued well on into the night taking advantage of Zurich's fine gastronomy and beverages. From CERN, I attended along with Belmiro Moreira and Daniel Lobato.
Thomas M. Bohnert gave an overview of the ICCLab (details at http://www.cloudcomp.ch/ ) which covered the role of the InIT Cloud Computing Lab investigating cloud technologies. Some of the lightning talks later cover areas of research.
CERN and OpenStack
I gave a rapid overview of OpenStack architecture, news from the summit, the foundation structure and use of OpenStack at CERN. The tools and procedures in use in the CERN computer centre are being adapted to be more sustainable and efficient by engaging and adapting open source solutions. Slides are available here .
It was a lot fo ground to cover in 30 minutes! Given that the audience ranged from relative novices in OpenStack up to those deploying, it is also a wide range of details to cover in a single talk.
Amongst the questions were what is the impact of moving towards clouds on the current MONARC grid model as used by the WLCG. One of the common challenges for CERN is how to solve the "Changing the engine while the plane is flying" problem. Even though there is an extended shutdown of the LHC during 2013/2014 in order to ugrade to higher energy, the physics analysis of the data taken in the first run from 2010 to 2012 will continue. Any major architecture changes on the grid must be performed in a staged fashion as there are over 130 sites and 200,000 servers to consider so big bang migrations are not realistic. Thus, the approach we are taking is one of running the grid services on top of the CERN private cloud. This allows us to gain improvements in efficiency via more dynamic workload placement and simplified operations procedures without needing to break compatibility. As the users of the service wish, resources can be moved between standard batch services on the grid and those available as VMs on the cloud.
Following up from this question was whether the other centres in the WLCG would be following the same approach. Each tier-1 is an independent organisation and can make its own choices within the framework of the WLCG. CERN is just one of the sites and each of the sites are able to make their own decisions based on individual requirements. Research into areas such as federated clouds is ongoing but no decisions for production federated clouds have been made.
Developing OpenStack APIs
Thijs Metsch from Intel gave a lightning talk on developing OpenStack APIs. Since OpenStack is a modular architecture, it is possible to plug in additional API functionality. This process is made more straightforward by the use of WSGI and python. Examples of this already implemented are the OCCI API and others such as CDMI are also coming along. Cloudscaling recently announced their implementation of the Google Compute Engine API for OpenStack. Thus, the choice of API can be made independently from the choice of IaaS technology.
Thijs demonstrated how this could be done by creating a simple python RESTful API and showed us the python code to implement it.
ICCLab Proof of Concept
Fabrice Manhart from the ICCLab presented the ICCLab proof of concept openstack cloud. ICCLab use Foreman and Puppet, as do CERN, and have good experiences with this combination. Being able to re-install their 15 hypervisors in 15 minutes is a typical case.
Swiss Academic Cloud Proof of Concept
Dean Flanders from Friedrich Miescher Institute for Biomedical Research /Swing presented their OpenStack based cloud which is testing out new computing models in a research environment. They selected OpenStack for its flexibility (and price :-). Their cloud allows researchers to provision their own machines rapidly. Amongst the areas they are investigating is the use of flash storage, aiming to reach 100TB SSD capacity. This approach follows work in the SDSC on Gordon which is a 16K cores 300TB flash based facility using OpenStack and ScaleMP.
While it is still early days for OpenStack, they are looking into new areas such as cloud bursting and workload balancing.
OpenStack at SWITCH
SWITCH provide facilities for the universities and academic organisations in Switzerland. Joel Casutt presented their early work to extend their existing services to include cloud services based on OpenStack. They are currently building up the hardware aiming to provide a free pilot service in April. During the summer, this experience will be reviewed along with the charging structure to be used going forward.
Wibke Sudholt from Cloudbroker presented their open source broker solution. Within the SCIentific gateway Based User Support (SCI-BUS) EU FP7 project, CloudBroker has extended its CloudBroker Platform so that it now also allows access to Amazon and IBM public clouds as well as to private cloud infrastructures based on OpenStack and Eucalyptus.
While the initial aims were to use the EC2 API in OpenStack, there were some missing features and functionalities such as auto-deployment so it was necessary to use some of the OpenStack APIs to achieve full function. Using OpenStack clouds in universities allowed this porting to be done without having to set up their own OpenStack instance. The XML support was also not as good as JSON in some areas.
Software Defined Networking at ICCLab
Phillipp Aeschlimann from ICCLab presented the current status of software defined networking and their investigations as part of ICCLab. There is a lot of interest in the OpenStack community around these features and Quantum functionality covers many features.
Even in a country with a small population such as Switzerland, there are many areas of OpenStack work. There was only time for 6 lightning sessions of 10 minutes but we are now planning the second meeting to allow those who could not show their work. Per-capita, the Swiss user group attracted 5 times more people that the Chinese user group meeting (where 3,000 people attended).
Many of the attendees were investigating/testing proof of concepts. The number of production deployments are small amongst those attending. As this is often the first investigations into cloud services, this is not too surprising. Many of the deployments were using Puppet as well.
During the evening, we discussed the competition for talent. Given that many CERN people are on short term contracts, the interest in OpenStack and Puppet skills makes for a good skill on their CVs.
A recent blog on comparing Google trends follows similar lines. The interactive report is here . Unfortunately, there is not enough data from Switzerland to determine the interest in different cloud products.
Overall, there was a lot of enthusiasm but this was balanced with caution that comes from uncovering some of the difficulities to understand clouds and how best to deploy OpenStack. The most frequent question at the end was "When will we meet up again?".