[Script Info]
Title: 
[Events]
Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,I am Nicolas Dandrimont.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,I am going to talk to you about a year of fedmsg in Debian.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,We had a problem before with infrastructure in distributions.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Services are bit like people.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,There are dozen of services maintained by many people
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,and each of those services has its own way of communicating with the rest of the world
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Meaning that if you want to spin up a new service
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,that needs to talk to other services in the distribution
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,which is basically any service you want to include
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,you will need to implement a bunch of communication systems
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,For instance, in the Debian infrastructure
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,we have our archive software, which is dak,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,that mostly uses emails and databases to communicate.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,The metadat is available in a RFC822 format with no real API.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,The database is not public either.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,The build queue management software, which is wanna-build,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,polls a database every so often to know what needs to get built.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,There is no API outside of its database
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,that isn't public either
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Out bug tracking system, which is called debbugs,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,works via email, stores its data in flat files, for now,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,and exposes a read-only SOAP API.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Our source control managament pushes in the distro-provided repos on alioth
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,can trigger an IRC bot or some emails
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,but there is no real central notification mechanism.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,We have some kludges that are available to overcome those issues.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,We have the Ultimate Debian Database
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,which contains a snapshot of a lot of the databases that are underlying the Debian infrastructure
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,This means that every so often,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,there is a cron that runs and imports data from a service here, a service there.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,There is no realtime data.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,It's useful for distro-wide Q&A stuff because you don't need to have realtime data
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,But when you want some notification for trying to build a new package or something
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,That doesn't work very well
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,and the consistency between the data sources is not guaranteed.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,We have another central notification system which the package tracking system
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,which also is cron-triggered or email-triggered
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,You can update the data from the BTS using ??
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,But you can subscribe to email updates on a given package
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,But the messages are not uniform,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,they can be machine parsed.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,There are a few headers but they are not sufficient to know what the messages are about.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,And it's still not realtime.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,The Fedora people invented something that could improve stuff which is called fedmsg.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,It was actually introduced in 2009.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,It's an unified message bus that can reduce the coupling between different services of a distribution.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,That services can subscribe to one or several message topics, register callbacks and react to events
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,that are triggered by all the services in the distribution.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,There is a bunch of stuff that are already implemented in fedmsg.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,You get a stream of data with all the activity in your infrastructure which allows you to do statistics for instance
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,You decouple interdepent services because you can swap one thing with another
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Or just listen to the messages and start doing stuff directly without having to ?? a database or something.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,You can get a pluggable unified notification system that can gather all the events in the project and send them by email, by IRC
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,on your mobile phone, on your desktop, everywhere you want.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Fedora people use fedmsg to implement a badge system
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,which is some kind of gamification of the development process of the distribution
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,They implemented a live web dashboard
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,They implemented IRC feed.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,And then they als go some bot bans on social networks because they were flooding
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,How does it work?
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Well, the first idea was to use AMQP as implemented by qpid
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Basically, you take all your services and you have them send their messages in a central broker
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,and then you have several listeners that can send messages to clients.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,There were a few issues with this.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Basically, you have a single point of failure at the central broker
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,And the brokers weren't really reliable.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,When they tested it under load, the brokers were tipping over ??
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,The actual implementation of fedmsg uses 0mq.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Basically what you get is not a single broker.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,You get a mesh of interconnected services.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Basically, you can connect only to the service that you want to listen to.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,The big drawback of this is that each and every service has to open up a port on the public Internet
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,for people to be able to connect to it.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,There are some solutions for that which I will talk about.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,But the main advantages is that you have no central broker
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,And they got like a hundred-fold speedup over the previous implementation.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,You also have an issue with service discovery
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,You can write a broker which gives you back your single point of failure
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,You can use DNS which means that can say "Hey I added a new service, let's use this SRV record to get to it"
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Or you can distribute a text file.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Last year, during the Google Summer of Code, I mentored Simon Choping
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,who implemented the DNS solution for integration in fedmsg in Debian.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,The Fedora people as they control their whole infrastructure just distribute a text file
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,with the list of servers that are sending fedmsg messages
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,How do you use it?
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,This is the Fedora topology.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,I didn't have much time to do the Debian one.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,It's really simpler. I'll talk about it later.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Basically, the messages are split in topics where you have a hierarchy of topics.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,It's really easy to filter out the things that you want to listen to.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,For instance, you can filter all the messages that concern package upload by using the dak service.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Or everything that involves a given package or something else.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Publishing messages is really trivial.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,From Python, you only have to import the module,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,do fedmsg.publish with a dict of the data that you want to send
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,And that's it, your message is published.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,From the shell, it's really easy too.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,You just have a command called fedmsg-logger that you can pipe some input to
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,And it goes on the bus, so it's really simple.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Receiving messages is trivial too.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,In Python, you load the configuration
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,and you just have an iterator
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,(video problems, resume at 10:10)
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,was a replay mechanism with just a sequence number
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,which will have your client query the event senders for new messages that you would have missed
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,in case of a network failure ??
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,That's how basically the system works.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Now, what about fedmsg in Debian
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,During the last Google Summer of code, a lot happened thanks to Simon Chopin's involvement
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,He did most of the packaging of fedmsg and its dependencies
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,It means that you can just apt-get install fedmsg and get it running
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,It's available in sid, jessie and wheezy-backports
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,He adapted the code of fedmsg to make it distribution agnostic
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,So he had a lot of support from upstream developers in Fedora to make that happen
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,They are really excited to have their stuff being used by Debian or by other organizations
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,?? fedmsg was the right solution for event notification
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,And finally, we bootstrapped the Debian bus by using mailing-list subscriptions
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,to get bug notifications and package upload notifications
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,and on mentors.debian.net which is a service I can control, so it's easy to add new stuff to it.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,What then?
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,After the Google Summer of Code, there was some packaging adaptations to make it easier to run services based on fedmsg,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,proper backports and maintainance of the bus
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Which mostly means keeping the software up-to-date
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,because the upstream is really active and responsive to bug reports
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,It's really nice to work with them
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Since July 14th 2013 which is the day we started sending messages on the bus
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,we had around 200k messages split accross 155k bug mails and 45k uploads
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,which proves that Debian is a really active project, I guess
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,[laughs]
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,The latest developments with fedmsg is the packaging of Datanommer
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Which is a database component that can store messages that has been sent to the bus
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,It allows Fedora to do queries on their messages
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,and give people the achievements that they did like "yeah, you got a hundred build failures"
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,or stuff like that [laughs]
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,One big issue with fedmsg that I said earlier is that Debian services are widely distributed
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Some of the times, firewall restrictions are out of Debian control
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,which is also the case of with the Fedora infrastructure
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,because some of their servers are hosted within Redhat
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,and Redhat networking sometimes don't want to open firewall ports
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,So we need a way for services to push their messages instead of having clients pull the messages
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,There is a component in fedmsg which have been created by the Fedora people which is called fedmsg-relay
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Which basically is just a tube where you push your message using a 0mq socket
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,and it then pushes it to the subscribers on the other side
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,It just allows to bypass firwalls
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,The issue is that it uses a non-standard port and a non-standard protocol
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,It's just 0mq so it basically put your data on the wire and that's it.
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,So, I am pondering a way for services to push their messages using more classic web services
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,You will take your JSON dictionary and push it by POST through HTTPS
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,And then after that send the message to the bus
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Which I think will make it easier to integrate with other Debian services
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,This was a really short talk
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,I hope there is some discussions afterwards
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,In conclusion, ??
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,I am really glad ??
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,For the moment, it's really apart from the Debian infrastructure
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,So the big challenge will be to try to integrate fedmsg to Debian infrastructure
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Use it for real
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,If you want to contact me, I am olasd
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,I am here for the whole conference
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,If you want to talk to me about it, if you want to help me,
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,I am a little bit alone on this project, so I'll be glad if someone would join
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,I'll be glad to hold an hacking session later this week
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Thanks for your attention
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,[applause]
Dialogue: 0,9:59:59.99,9:59:59.99,Default,,0000,0000,0000,,Was it this clear?