- Developer Tools
Phil Whelan, July 31, 2014
Our regular MC, Chris Ferris from IBM, was absent this month, so Michael Maximilien ("dr.max") from IBM kicked off this month's Cloud Foundry Community Advisory Board (CF-CAB) Meeting and kept things on track.
Mike Maxey from Pivotal gave an update on the where the Cloud Foundry foundation is in its formation. During the summer, and especially after the Cloud Foundry Summit, there has been a drop in momentum for this.
Mike said that we are still targeting the end of the year for standing up the foundation. Pivotal has been working with the Platinum members of the foundation to define the guiding principles and the high ideas around how governance might work. They are also looking at how projects will come together.
By October they plan to have all the requirements and documents finished for the non-profit entity. Then by November they hope to do an announcement of the Foundation.
They are now at a point where they can start working on the formal documents. Mike noted that this will obviously take a substantial amount of time.
Mike said they are hoping to share ideas and content with the Cloud Foundry community in the next one or two months.
At the Cloud Foundry Summit a total of 35 foundation members were presented. Since then a few more companies have shown interest in joining the foundation, but Mike said that it is unlikely these will be announced until November. Mike gave two reasons for waiting until November to announce any new members:
To have enough new members to have a meaningful announcement (15+)
To show progress with the foundation beyond members intending to join. The November time-frame is due to the planned timing of finalizing the non-profit organization.
Joanne Muszynski from IBM asked if the regular foundation board meetings will start in November, or whether they are already happening. Mike confirmed that those meetings have not yet started and they will likely start in November.
Joanne also said that she had heard that it was becoming harder to form these kinds of non-profit foundations and wondered if this was causing any delays. Mike said that it was becoming more difficult to form a classic non-profit organization, but the Cloud Foundry foundation is going to use the same model as the OpenStack Foundation.
While Mike does not foresee any issues, he did say there are some hoops to get through. For instance, you need to publicly display all of your by-laws and documents for 45 days before the non-profit status will be granted by the government.
Heather Meeker, the foundation attorney, has looked at the process. Also the Pivotal attorneys have looked at it. Therefore, Mike thinks it should be quite straight-forward.
Pivotal PM Updates
James Bayer gave some updates to the Product Managers within Pivotal.
- Dmitriy Kalinin, who has been a developer on BOSH will now become a the Product Manager of BOSH.
- Greg Oehman, who was formerly the BOSH Product Manager, is now the CLI Product Manager.
- Onsi Fakhouri, who has been the lead engineer on Diego, is becoming the Diego Product Manager. Onsi gave a great presentation on Diego at this year's Cloud Foundry Summit, which is a must-see for anyone wanting to get up-to-speed with what Diego is.
- Alex Jackson is now doing Product Management for Logging and Metrics.
These people will be points of contact for the various Cloud Foundry projects.
Services - MySQL
A version 9 release has just been made available of the MySQL service.
Shannon Coen, the Product Manager for Services at Pivotal, said they are looking to make the MySQL service more robust in terms of high-availability. The v9 release is a single MySQL server and only supports one shared plan. The next release of the MySQL service will support multiple plans.
Some significant architectural changes have been made for this upcoming release, which they are targeting for within the next month. The biggest change is that the MySQL server has been replaced with MariaDB. Also, MariaDB Galera clustering has been introduced, which is a synchronous multi-master cluster for MariaDB.
Initially this service will not be highly-available. All requests will be proxied via a single HAProxy server. Although, the subsequent release will look at adding an additional HAProxy server and implementing fail-over.
Services - Riak
Shannon said that issues have been resolved with the Riak service, which involved data not being written to a persistent volume.
Syslog forwarding has also been introduced.
A new v4 final release of this Riak CS service is now available.
Services - Plans
Discussing the Services road-map, Shannon told us that they are planning to add support for updating or changing a single service instance. The initial focus of this will be to change the service plan.
Shannon said that he will be working with Greg Oehman (the new CLI Product Manager at Pivotal) on the related new CLI functionality to update a service instance.
This is will then lay the groundwork for being able to change other things about the service instance.
Generally, the MySQL (via cf-mysql-release) and Riak CS (via cf-riak-cs-release) services are deployed separately to Cloud Foundry (via cf-release). Shannon said that the release engineering team at Pivotal are looking at doing composite deployments that include these services. This is to enable the use of the highly-available MySQL service (see above) to back other components, such as the Cloud Controller and UAA. Likewise, Riak CS could be a potential replacement to the NFS blob store.
These two open-sources services, MySQL (MariaDB) and Riak CS will also provide out-the-box HA services with this composite release.
James Bayer said that this would be a big milestone if we could have the data-stores of key Cloud Foundry components be highly-available out-of-the-box.
Greg Oehman from Pivotal started his update by warning that he had only been on the job for 3 days.
He said that the internalization work is practically complete. There are just a few more tasks to wrap up.
Some work has been done for Services, which involved service plan access, visualizations and filtering.
For internationalization, the client now has supports for five languages, which Greg said is a great start. He said that the next one might be Catalan, which Ferran Rodenas ("Ferdy") from Pivotal has requested.
Greg intends to re-ignite the conversation around plugins and find the best path to move forward with this. He said they want to do this soon and do it rapidly.
Logging and Metrics
Alex Jackson from Pivotal said that they are working on eliminating the parts of Loggregator that are not highly-available. For instance, the data pipeline path is not redundant going through the Traffic Controller. Even if you do have multiple Traffic Controllers, it will still only use one of them.
They are using a similar pattern to Diego, by putting the active components into etcd. This allows for monitoring the active health of these components and traffic will be routed only to the currently active components within the system.
Alex said that they will continue to work on making this new architecture stable over the next month. They will also work on including the beginnings of the Loggregator metrics.
The new Loggregator metrics will be provided via new Loggregator API endpoints, so will not affect existing API endpoints and will therefore be backwards compatible with the CLI's existing integration with Loggregator.
Alex said his team would like to promote the dropsonde repository out of the Cloud Foundry incubator and into the main Cloud Foundry repos. This defines is the emitter format for the beginnings of the metrics used by the router which the Loggregator intends to also use.
James Bayer gave the update from the Runtime team.
v175 was the first release to include fully implemented CF Security Groups. James said that his allows administrators to set up the outbound network policies for applications. This can apply at staging or runtime.
There are a core set of policies that apply to all applications and then additional security groups can be applied to individual Spaces.
For example, a Space named "Production" might get exclusive network access to talk to the "Production" database. An application pushed into Cloud Foundry gets the general security access provided by the system, plus specific security access provided by the Space policies.
James said that this improves the multi-tenancy of the system and allows the treating of individual spaces as individual tenants.
James mentioned a new feature that gives the ability to restart an individual application instance. Previous to this there was only the ability to do a restart at the application level, which would involve fully stopping the application deploying new instances of all of the application containers.
James said that this is a minor feature, but "kinda neat". He gave the example of having 5 running instances and seeing an issue with just one of them. Now you can restart that one instance without downtime to the entire application.
A new Ubuntu 14.04 ("Trusty") stem-cell has been built for cf-release and works with v176. You can still also use Ubuntu 10.04 ("Lucid") with v176.
James warned that going forward support for Lucid stem-cells will be dropped. He said that there has been some requests to continue producing Lucid stem-cells, but the burden of supporting both Lucid and Trusty stem-cells long-term is too much for the Pivotal team. This is due to the cost of providing continuous integration testing on these components. The matrix of things that they are testing is already very complicated.
While nothing intentionally will be done to break support for Lucid, thorough testing will only be done on Trusty. Therefore, they will not be able to certify support for Lucid.
Colin Humphreys from CloudCredo asked whether the application containers filesystem will also move to Trusty, or whether it will remain on the Lucid filesystem. He also suggested that this might be option via the "--stack" parameter.
James said that for now they are sticking with the Lucid filesystem for the Warden containers. The main reason given was lack of experience with the Trusty filesystem within containers and all the Cloud Foundry buildpacks currently work with the Lucid filesystem. He said that it is a long-term, but unscheduled, plan to move to the Trusty file-system for containers.
James said that the Runtime team are currently working on "Space Quotas" and "Organizational Buildpacks". Both of these provide a way to provide these existing pieces of functionality at a greater level of granularity.
Other things briefly mentioned by James were...
Feature flags. These are being added to enable and disable features of the platform using the Cloud Controller API. You can use this to turn off a feature dynamically and stop it from working.
Role-based access control. James said that currently only "pre-canned" roles are available. This will allow Cloud Foundry administrators to define new roles that have specific access to the Cloud Foundry models.
James said that the designs for these are not finalized and that you can follow the mailing-list to see discussions on how these problems are being tackled.
David Stevenson from Pivotal gave an update on Notifications, before running off for his flight.
David said they now have a synchronous application that obeys the Notifications API and allows the components of Cloud Foundry to send notifications to either individual Users or all Developers of a specific Space.
The application uses templates, which can be modified by whoever deploys it.
They plan to announce this being "ready for use" within the next week, after which it will pushed to the Services teams for integration.
James Bayer proxied the update he got on Diego from Onsi Fakhouri of Pivotal.
Diego is continuing to be hardening and tested. Pivotal is now using an internal deployment that co-locates Diego in their production environment. They are able to opt-in to using Diego via environment variables. This covers the staging and running of applications.
James said that there are still "plenty of gaps". There is currently no health management in Diego - if your application crashes in Diego, it would not be restarted. Some application statistics have not been implemented. There is no read-only file-system, like with the directory server.
IBM and CloudCredo are helping with adding support for Docker containers in Diego. This would work by end-users specifying a Docker image instead of a Buildpack when deploying an application. Once the container is running, it is treated like any other application in Cloud Foundry, said James. For instance, there is no change to routing requests or managing of Loggregator logs.
The work for Docker support in Diego started a couple of weeks ago and there is a design document you can read and comment on. IBM developers are working with Pivotal on this at the Pivotal offices and they expect to see progress on this over the next few months.
James gave a brief update on BOSH.
The Trusty stem-cell contains the go-agent, which means that Pivotal's production environments are now using the go-agent. Therefore they believe this agent to be production ready.
James said that there is a lot of focus on the external CPI's from Pivotal's BOSH team. Usability is also being constantly improved.
UAA will shortly get a new Product Manager.
Work continues on OAuth.
James said that the Java Buildpack will shortly be moving to Java version 8. This will be the default Java runtime, but it will still be possible for users to change this to run application under Java 7.
Python, PHP, Go, Ruby and Node.js are being evolved to work better off-line and also to be more automated in the way upstream commits are taken from Heroku. These will be certified to work with Cloud Foundry in "offline" mode, then packaged up in cf-release.
Thanks to Michael Maximilien from IBM for running the meeting and also sending me his notes to assist in the write-up.
The Cloud Foundry Community Advisory Board started the session focussing on the Cloud Foundry Summit held in San Francisco earlier this month. Overall, the feedback about the conference was very positive. Cornelia Davis summed it up best, "The conference was really great...the energy was tremendous. The tidal wave that you could feel from the momentum around Cloud Foundry and the Cloud foundry community was palpable there." A big thank you to everyone who organized the event!
Chris Ferris of IBM started off today's Cloud Foundry Community Advisory Board meeting reflecting on the recent OpenStack conference. "One of the things I observed is that it's definitely starting to trend towards discussions around containers, and Cloud Foundry and OpenStack seem to be like hand-in-glove in a lot of the conversations.". Renat Khasanshyn from Altoros added his comments on the OpenStack Summit in the chat that "Docker conversations in the hallways heavily leaned towards Cloud Foundry" and "OpenStack ecosystem turned the corner and becoming all in with Cloud Foundry."
Subscribe to ActiveState Blogs by Email
Share this post:
Tags: bosh, buildpacks, CAB, cli, cloud foundry, cloud foundry foundation, diego, docker, mysql, riak, services