Home Home > Infrastructure
Sign up | Login

Archive for the ‘Infrastructure’ Category

Tumbleweed waits for workers

February 17th, 2016 by

Tumbleweed-black-greenopenSUSE’s rolling distribution Tumbleweed goes through automated tests before a snapshot is released and heavily relies on openQA for the process of Tumbleweed to create regular snapshots.

The last snapshot was 20160209, which produced new grub2, python, YaST and GTK3 packages.

The automated testing of openQA is currently running with only two workers left instead of the usual 10. The remaining workers are largely overloaded and can’t cope with the workload to produce new snapshots.

Various solutions are being evaluated to get new workers for openQA, which includes borrowing machines from other SUSE owned instances. Thank you SUSE!

The team has opted to hold back creating new snapshots until more workers for openQA become available. Once the team resumes with staging, testing and releasing, which is expected for a 20160217+ snapshot, Tumbleweed fans can expect updates to Firewalld, CMake 3.4.3, GStreamer 1.6.3 and Plasma 5.5.4.

OBS welcomes new lambkins

January 27th, 2016 by

The openSUSE build service becomes more and more a victim of his success: building constantly more than 300,000 packages for more than 43,000 developers needs really a lot of build power! And build power means not only CPU! It includes everything that you can expect from an IT infrastructure:

Old hard-drives from OBS-workers

Old hard-drives from OBS-workers

  • CPU power
  • RAM (the more, the better)
  • Storage (temporary local, on the clients and also to store and distribute the results)
  • Network
  • electic power (and cooling, and maintenance, and manpower to maintain the hardware, …)

Thankfully our main sponsor SUSE allowed us now to buy some new hardware to replace some of the old machines that build software packages for over ten different distributions all day long.


Hardware problem: rsync.opensuse.org down

January 11th, 2016 by

rsync-logoWhat a start in the new year: the server running rsync.opensuse.org died with two broken hard disks at 2016-01-10.

As the hardware is located in the data center of our sponsor IP Exchange, we apologize for the delay it will take to fix the problem: we need not only the correct replacement hard drives, but also a field worker at the location who has the appropriate permissions and skills.

During the downtime (and maybe also a good tip afterward), please check on http://mirrors.opensuse.org/ for the closest mirror nearby your location that also offers rsync for you.

Weekend Wikithon to refresh content

October 19th, 2015 by

hatLeap comes out in 16 days, but before it does, openSUSE will have a Weekend Wikithon Oct. 24 and Oct. 25 to update, delete and refresh content on the openSUSE wiki.

Contributions will be gauged and four contributors will receive an openSUSE Leap hat that have the most contributions between  Oct. 24 and Oct. 25 in the following four categories:

  1. Contributor who updates the most content on the features page for the upcoming release of Leap 42.1
  2. Contributor who updates the most screenshots on the screenshots page to Leap 42.1
  3. Contributor who makes the most contributions to the 42.1 portal
  4. Contributor who deletes the most irrelevant content on the openSUSE wiki.

Communication for the event will take place on the Freenode IRC opensuse-marketing channel.

Testing Leap milestone in openQA

July 15th, 2015 by

Screenshot from 2015-07-15 15:11:21Developing Leap 42.1 is happening quickly and it was announced yesterday that the milestone was being built.The first milestone will hopefully be released this week. Leap is going through its testing and the importance of openQA (Quality Assurance) in this development process can not be understated.

openQA is used for testing an operating system, finding and files bugs and provides fully automated testing to ensure a distribution work correctly with clean functionality.

The wonders of openQA determine if a build is good and triggered errors allow testers and developers to see errors quickly and determine a cause.

Feel free to look at the test of the Leap milestone and use the hints below to navigate through openQA.

See what Richard Brown, openSUSE chairman, had to say about Leap and openQA on Reddit.

How To: Use openQA to find and file bugs (more…)

Planned downtime 2015-06-09: Mailing Lists

June 5th, 2015 by

Update: The migration of the openSUSE Mailing Lists has been finished successfully. If you encounter any issues, please let us know by mail to admin@opensuse.org.

On Tuesday 2015-06-09, from 09:00 to 11:00 UTC, the machine that hosts the
openSUSE Mailing Lists will be offline. During that
time, sending or receiving mails to the  openSUSE mailing lists, or viewing
their archives will not be possible.  All the mails that will be sent during
the downtime will be  delayed.
The  reason is that the old machine is on an old distribution, and running  out
of resources. We will migrate the service to a new virtual machine,  that will
integrate it to a new configuration management infrastructure.

We’ll send a followup announcement with the final status as soon as we finish
the migration.

On behalf of the openSUSE Administration team

OSEM in 10.1 simple steps

April 23rd, 2015 by
osemnewsby Alex Vennos

What is OSEM?

OSEM is an event management web application, tailored to the needs of FOSS conferences. You can visit http://osem.io/ to find out more about it.

You can contribute too!

The guide is based & tested on openSUSE 13.2 and it will help you get started with your development right away!

How to install OSEM

Step 1. Install Ruby & Bundler (version >= ruby 2.1.2)

sudo zypper in ruby rubygem-bundler

Step 2. Install git to your system

sudo zypper in git

Step 3. Clone the repository locally to your machine (more…)

Sometimes you need some luck (was: build.opensuse.org downtime)

April 24th, 2014 by

A morning you love as admin: starting with one single disk in your storage array failing, ending up in a whole array crashing. (more…)

News from your openSUSE admins

April 12th, 2014 by

Heartbleed and openSUSE infrastructureHeartbleed Logo

As people started to ask, we checked all openSUSE servers and can confirm that none of them is affected by the heartbleed bug.

For those users running openSUSE 12.2 and 13.1, we can just repeat what we always pray: please install the latest official updates provided by our glorious maintenance team.

RSYNC and rsync.opensuse.org

The server behind rsync.opensuse.org is re-installed now and already providing packages via HTTP again.

But we faced an issue with the automation that creates the content of the “hotstuff” rsync modules: normally a script analyzes the log files of download.opensuse.org and arranges the content of these special rsync modules to provide always the most requested files, so our users have a good chance to find a very close mirror for their packages. But currently the script is not producing what we expect: it empties all those hotstuff modules. As the core developer behind this script comes back from vacation on Monday, we hope he can quickly fix the problem. For now we disabled the “hotstuff” modules (means on rsync.opensuse.org: we disabled rsync completely for now) to avoid problems.

If you want to sync packages to your local machine(s) via rsync: please pick a mirror from our page at mirrors.opensuse.org providing public rsync.

New hardware

All the racks of the OBS reference server

All the racks of the OBS reference server

You may have noticed already that the openSUSE team installed a new version of openQA on the production server. An additional news item might be that this new version has seen also new hardware to run faster than ever.

But not only openQA, also the database cluster behind download.opensuse.org has seen a hardware upgrade. The new servers allow to run the database servers as virtual machines, able to have the whole database structure stored in RAM (you hopefully benefit from the faster response times on download.opensuse.org already). And the servers still have enough capacity left, so we can now also visualize the web servers providing the download.opensuse.org interface. We are currently thinking about the detailed setup of the new download.opensuse.org system (maybe using ha-proxy here again? maybe running mirrorbrain in the “no local storage” mode? …) – so this migration might take some more time, but we want to provide the best possible solution to you.

Admins on openSUSE Conference

These year, three of our main European openSUSE administrators are able to attend to the openSUSE Conference in Dubrovnik:Geekocamp

  • Markus Rückert
  • Martin Caj
  • Robert Wawrig

And they will not only participate: instead they are providing talks and help with the infrastructure and video recording of the venue. So whenever you see them: time to spend them a drink or two :-)





The new generation of openQA hits the production server

April 4th, 2014 by

Bad news for the bugs: the new version of openQA is ready for prime time. Everybody following the blog of the openSUSE Team @ SUSE or the Factory mailing list during the last months, should be aware of the ongoing work to improve openQA and to promote it into a key component of the openSUSE integration process. Finally the new openQA is ready for public production environments, so thanks to the collaboration between the openSUSE Team and the original developers of openQA –Bernhard M. Wiedemann and Dominik Heidler– it’s finally deployed and accessible at openqa.opensuse.org

This new version brings a lot of changes at many levels, but probably the most relevant difference is the approach for tests execution: instead of running every step sequentially and comparing the needles at the end, the new version evaluates the status several times per test, deciding what to do next based on that status or aborting the whole tests as soon as a critical error is found. This approach enables both a better usage of the resources and more precise results.

This enhanced control of the execution and the results, alongside other improvements, makes possible to extend the scope of openQA. Tests of Factory isos are still there and running. But apart from them, you can see test results for the so called "staging projects", used to merge potentially dangerous packages. Generally speaking, you can just browse the test results and see what state is Factory in and how dramatic changes are about to happen.

Fuzzy matching in action: ignoring the floppy icon

Another main new feature is the use of fuzzy area matching for interpreting test results. That means much less false positives. Tests do not break that often and that easily. There is also a nice interface to figure out what failed. Try going to some failed test, selecting a needle and dragging the vertical yellow line. Pretty neat, isn’t it? You can also check how the test is written and what is it looking for. Feel free to play with it, enhance the current tests and needles and submit them via GitHub ;-)

There are even more changes, not directed towards users, but improvements in the interface that service operators use to set things up, including users management, job control or a new REST-like API. These will not affect most of you directly, just indirectly by making operators job easier.

So go ahead, play with it and if you want to help, sources are on github and we even have some easy hacks in progress.o.o to ease you into the development ;-)