Music Herald: Hello everybody, we are ready to get started we have Lucas and Amir here and they want to give us a quick introduction of a project from the wikimedia foundation called "cloud services" and how it might be may be useful to all of us. So let's give a round of welcoming applause to Lucas and Amir. Applause Lucas: Thanks! yea, hello. So "wikimedia cloud services" is basically this big collection of all kinds of different things which are useful if you want to do taking your things in the wikimedia universe like with wikipedia or other projects and you get them free of charge or you can just use them and the only requirement is that you use them for something that's a kind of relevant to the mission of wikimedia of promoting free knowledge and that kind of stuff and it's kind of split into the things that you can do with your regular wikimedia account which any registered user can do and then there's also things you need a special account for on a different system called wiki tech and Amir is going to talk more about those later but first let's just look into some of the things you can do with your regular wikimedia account. And if you want to follow any of these links there's a shortcut here. I was about to switch the next tab, so let's just stay here for a few seconds yeah. So the first thing is the API sandbox which is if you want to use the MediaWiki API to figure out what you have on a page or to make edits or any kind of stuff. The API sandbox is a special page that's really useful to find out how to use the API for example here's all the different actions I can use that say query is the kind of general catch-all action that's here and then I get down here a list of all the parameters I can use with queries such as: I want to have all the user info and what kind of user info do I want? I want options, blablabla. I would like to have some different format versions. So it gives you all these nice inputs for figuring out exactly how to use the API what's valid what's not valid and then you can make the API request and there you get a response and we can't read anything because it's zoomed in way too much. But it's very helpful when trying to use the API and then in the end here you can see what you need to do in your own code to make the same API request. And for anything that you can't do with the normal API - so if you want to do some kind of more expensive analysis - you can often do that with Quarry, which is a tool that lets you write SQL queries against databases that are almost like the ones in production like you don't have user passwords and stuff but you'll have all the database tables with page metadata and connections between them and the logs and all kinds of stuff and you can just write your SQL here send it and you get the results for example here's the number of lexemes published a days so it's some kind of selecting from the page where the namespace is the lexeme namespace and grouping that by the date and then we get something like all the way down to September which is apparently when I ran this query there were here there were 116 lexemes created in this day. Or here someone had a list of edits to JavaScript and CSS pages on Hungarian Wikipedia so you can run these queries against any Wiki you like, like this here in wikipedia one. And if you can't get by with just SQL what you also have is this thing called Paws, which gives you a Jupiter(?) instance if you've heard of that you can basically write your own Python code here and do it in a very convenient way because there's all kinds of auto-completion and helpful things. So i can just try to copy this and run the code (then I needed a new cell below it… there we go, Thanks!) and if I type item I should get helpful hints what I can do with the item (if it's not hanging or something or the tab control space no oh there we go yeah) and it's also a very useful way to work with py- wiki-bot or you can also directly get normal shell here. And one thing (oops did I click and wrong thing? I would like to have oh no I don't want a bash notebook I want a new terminal that's what I want). And here you have for example database dumps in (where was it?) public/dumps/ something public again… So if you want to do some kind of analysis here on the data dumps you can get them here and then have all the computing that you want I guess to analyze the wiki more thoroughly and all of this is hosted in the Wikimedia Cloud for you and you don't need your own server or anything. Oh yeah I had two more examples of that, for example here: I use that too so there were a lot of items on Wikipedia where there was some encoding error, this should be an apostrophe like down here and instead it was this kind of I with an accent and I hacked together some ugly Java/Python code to make all of these edits and it was already logged in as well I didn't need to worry about logging in or having a password or anything. So it's a very convenient way to make edits as well. Or you can build something nicer here you can insert like markdown cells to explain what you're doing and how the code works and build nice notebooks like that, which are almost self-explanatory. And those are some of the things you can do just with your Wikimedia account and now Amir is going to talk about some other things. Amir: Thanks Lucas! So the thing that we can do is that maybe some of you like me think that doing thing in browser is for kids I need to do things in terminal I need to do connected system and then you can access for a wiki tech account which you can just make a wiki tech account in this place called wiki tech. (where is the li… no no but I do'… the main thing, the main list. yeah okay) And so in here so and then you make a wiki tech account and it gets approved quickly and then you get the shell and then you can just quickly go there (where is yer…) and you can go to this shell and just log in and then you have access to day a big set of nodes in the cloud and you can just do whatever you want. Also you have access to the core dumps and you have access to the replica database. Let me show it to you. [mumbling] So for example you can go to LS /public/dumps/public/wikidatawiki/ and then you get - oh there's like all sorts of time and everything that you want to, but if you also… you can do something else is that you can just do SQL wikidatawiki and then you go inside the wikidata's database, I mean it does you don't have the rights you can you cannot write to their replica because it's a replica and also it's sanitized so it doesn't have their like hash of user password and stuff like that but still you can do just select varies from recent changes limits five and yeah and then you get all of the things that you want you cannot even describe anything you want to directly into their system and then there is also we have something called the job grid so you can just put a crown and anything that you want to or just it's something run something directly and you goes to the a big note of cloud kubernetes and then just runs everything that you want to in its here there's a more information about it in here there's a like a long help that it says like oh I used to run this job and then job of what it does and you can get this so you just need to it's a bash command you can run any bash command and send it okay return me this output to this place and the other places one thing that you can do is also there's a web server that you can access everything directly so you can just put a PHP file there and into the Apache and then yet for example this is this is an example that we built together I think two two Christmases ago but this was like you can just see this is a piece before the source code is available and you just copy pasted that source code into like a directory and it was there and every time we click on it and you get most of the edits that happen on description wiki data that might be vandalism and we can fix it also a this is not just the only thing that you can do with this is that you can also put a Python flask application is this the file implants and then this can be just a Python application and you can just have the file there and also know JSON Java there's so many of them also you can have own database like I have something that has its own database for example quick categories in here has jobs that are here this is this tool for its own built-in database inside our select cloud services and its uses it just fine you can do that as well and also there's a cloud VPS that it doesn't do any kubernetes it just you can make a VPS of your own and then do whatever you want with it so for example and you get a project and you get the quota it's a slightly more limited but also you have access to the whole VPS you have sudo rights on it you can do whatever you feel like about it so we have like for example this project in here and it's called tools and then there's proxies and you can for example go into that instance and reboot it and do whatever you want and you can make new instance and look at your culture and look at everything else there and also you can also make it even a wiki on one of those cloud VPS systems which is for example we did in here in here if you look at it it's just a wiki and the difference is that for other ones for example for the vandalism dashboard you have tools that wmf labs org and then slash WD w VD which is the tool itself but in here we get our own subdomain so which will be wiki data - like seam that flew out the wmf labs org and you can even put all sorts of add subdomains for the wmf labs or as long it's not taken so you can build a media week instance instance or you can just complete a new software anything you can put a word processor who cares and then you can use it it's very simple your own thing and you can help lots of experience. Anything else? Lucas: I don't think so. Most important I would say is tool Forge to run your websites or if that's not enough for you cloud VPS and then you get your own VMware you can do absolutely anything you want as long as it matches those rules and stuff and I think that's it are there any questions? Herald: Hello thank you very much for the talk that was very quick so maybe anybody has a question here I'll give you my microphone to ask it. I don't see any hands nope okay I don't think we have questions but if you're just too shy to ask I think these guys always hanging around here around the wikipaka wiki so if you have anything you want to talk about you'll find them later okay then give a round of applause again for Lucas and Amir. Applause Music