Not sweet but salty, SUSE Manager 3 Technical Backstage Part I
Mar 16, 2016
Introduction
During the last year we have been working on SUSE Manager 3, the next release of SUSE’s Systems Management product, which includes additional capabilities around Configuration Management and Compliance. This article details this journey from the team’s perspective that may be of interest to product enthusiasts and developers.
Sweet did not last
In mid-2014 I wrote about SUSE Manager 2.1. It was an important release for us because at that point we became very active upstream, to the point that on a typical day, a considerable chunk of the open pull requests came from SUSE, including a refreshed mobile-enabled user interface.
But the world is changing, and management offerings are taking new innovative directions, especially around Configuration Management and Compliance. This trend was in several dimensions quite radical for our product, based on the Spacewalk project, which has been managing Linux systems with the same paradigm since 2008.
Deploying a management solution is a considerable investment for the customer: operation, training, processes, etc. Telling a customer you will be completely replacing their deployed solution because new trends have appeared means you are throwing all their efforts in the trash.
On the other hand, it is very easy to find excuses to rewrite software from scratch and there are well-known industry voices giving advice on the topic.
So are we up to the challenge?. Can we evolve SUSE Manager iteratively within this new world without endangering our customer investments?
Choosing a configuration management engine
After we decided that we would bring this new world into the existing SUSE Manager, we clearly didn’t want to write a configuration management engine ourselves: there were multiple opensource projects that were up to the task. We also understood that customers could be already invested into one of them, and that is fine: we were looking for the one that could be tightly integrated into SUSE Manager and had a design that matched our vision.
And here is where things got even less sweet: they got salty!
We ended up choosing Salt. The reasons are endless: From its vibrant community to the “aha!” moment you get when understanding its internals and seeing the vision behind the architecture. It is a complete toolkit to build datacenter automation, wrapped in usable tools with sane defaults that work out-of-the-box.
SaltStack platform or Salt is a Python-based open source configuration management software and remote execution engine. Supporting the “Infrastructure as Code” approach to deployment and cloud management, it competes primarily with Puppet, Chef, and Ansible. (Source: Wikipedia]
SaltStack software orchestrates the build and ongoing management of any modern infrastructure. SaltStack is also the most scalable and flexible configuration management software for event-driven automation of CloudOps, ITOps and DevOps. (Source: SaltStack)
We did not get the same impression from the other tools we evaluated. It was clear that Salt fit into the SUSE Manager present and future.
The fact that Salt implements configuration management (states) on top of the execution engine (state.apply), and that the executions modules are first class, we could offer almost all features we already had in SUSE Manager and naturally extend the Configuration Management part. On top of that, Salt is written in Python, and the current Spacewalk client stack and partly the server stack are also Python components.
Things only got better from there. During all this time we got really crazy about Salt. Our sysadmins are using it as well, and a community was formed internally that went beyond our product. Not only with sysadmins and engineers, but even our Product Manager writes beacons.
Suminator
The first serious dogfooding of Salt we did was the birth of Suminator. Silvio combined Vagrant with Salt to build a small tool that allowed us to build release and development versions of SUSE Manager and vanilla Spacewalk servers and clients in all possible code stream and operating system combinations. Very soon it became the tool of choice for developers to work on the codebase.
This tool is still tied to our internal build service and package repositories, but we hope we find time to make it useful to others as well.
If you want to play with openSUSE and Salt using Vagrant, I have published a repository that will get you started.
SUSECon 2015
The awesomeness of Salt started to click at various levels. The idea of orchestration built on the concept of Event-Driven Infrastructure plus the support for dumb devices culminated in the great SUSECon demo.
At SUSECon we showed that with the concept of reactive infrastructure; it was trivial to react to changes in configuration drift, in this case, using an inotify beacon that had “knowledge” about the managed files on that system, and make SUSE Manager react. On top of that we interacted with smart lights via proxy minions, just like you will have to do once the IoT takes over the world.
On top of that we showed that we could achieve all using the power of opensource that SUSE has been doing for almost 25 years. Support for Hue lights in Salt was already upstream by the time of that demonstration.
Refreshing our platform
Working on a new release means the opportunity to refresh the platforms and technologies you use, and to look for better alternatives for some of them.
- We keep rebasing and picking up enhancements from Spacewalk upstream.
- A mature codebase does not mean you should not get rid of code. E.g,: here is a pull request from the team to remove 30k lines of code that did not make much sense nowadays.
With the Salt and Compliance work there was going to be new code written, and that is an opportunity for choosing the right platforms and frameworks.
Integrating Salt into SUSE Manager
The first attempt was done as part of Hackweek 11. A protoype known as Saltwalk was born.
This protoype (a simple python reactor) helped figuring out what the bulk of the work would be, the non-trivial parts and what decisions we needed to take to move forward.
The basic architecture of a reactor that handles Salt events and interacts with Spacewalk was in place. What we needed now was a way for Spacewalk to interact with Salt.
salt-netapi-client
For the interaction of SUSE Manager with Salt, a Salt client library for Java was created, which allows to consume Salt functionality through salt-api.
Months after the original announcement, salt-netapi-client keeps being the best option available to interact with Salt from Java.
We pointed applicants to our open positions to salt-netapi-client as a challenge. Various contributors to the library became SUSE Manager team members!.
Becoming a Salt Master
When the decision of using Salt was clear, it was decided that we would do the integration code on the Java side of SUSE Manager, and so Saltwalk stayed as a protoype, and the functionality was implemented in Java.
At this point, SUSE Manager default installation was at the same time a full fledged Salt master server.
A consequence of this is that you can enjoy Salt on openSUSE out of the box. The Salt package is kept updated on Tumbleweed and Leap, which fits very well with the fact that openSUSE is available from various Cloud Providers e.g. #1 out of the box.
Registering minions
The next step was to make SUSE Manager aware of minions. First by registering them as they appeared (after salt-key --accept was done), so that they show up together with traditional systems:
After we had done work in order to retrieve the inventory data using Salt itself, the details page of minions was also available:
Once this was working we improved on it, allowing to operate the salt-key functionality directly from the user interface. Once a minion key needs to be accepted you would see it in the overview page:
And from there you can accept which ones to onboard:
Configuration Management with SUSE Manager
While you can patch, install packages in the same way you did with traditional clients, there are two main differences:
- What you do is instantaneous (unless you schedule for later)
- Instead of doing imperative actions (eg. install this package), you can also use states to define “what should be installed” in a declarative way using the power of States.
- You can write plain
slsdata in custom states
Additionally, SUSE Manager also has a higher level state user interface for packages. With this user interface you can search packages in the assigned channels.
Common states for organizations and groups
SUSE Manager allows to apply states from the State Catalog to Organizations and Groups. Every system belonging to those entities will be subject to those states.
Massive command execution
The Remote Commands page in the Salt section gives you a web based version of salt '*' cmd.run. You can preview which minions will be affected with the target before sending the commands and then see the results in real time:
Being part of the ecosystem
Making Salt an important part of SUSE Manager does not end there. In our industry being an Enterprise distributor of open-source software only works if you are part of it.
- SUSE already has around 600 commits from 5+ developers in Salt upstream
- The SUSE Manager team is hiring so that we can do more work upstream and help shape Salt’s future
- SUSE will be gold sponsor at Saltconf 2016
The future
As you can see, SUSE Manager with Salt is a powerful duo which allows you to continue managing your infrastructure with the same tool, be 100% backward compatible and start taking advantages of declarative configuration management and the orchestration capabilities of Salt, while keeping everything you have already deployed untouched.
We are very excited about the possibilities here, which will be guided by feedback from our customers and synergies we have with Salt and other SUSE technologies.
To be continued
Configuration Management is only one of the features that will arrive SUSE Manager 3. Expect to hear from the powerful Subscription Matching and Topology Awareness in future posts.