English subtitles

← Wheres the Data - Data Wranging with MongoDB

Get Embed Code
4 Languages

Showing Revision 2 created 05/25/2016 by Udacity Robot.

  1. Okay, so let's do a little exploring and think about the
  2. mechanics of this problem. The first thing I'd like to know, is
  3. how do I go about requesting the data I want from this
  4. website? Again, I'm thinking about how I get to a point where
  5. I can actually do this programmatically. So most web browsers give
  6. you the ability to actually inspect individual elements of a webpage, so
  7. in this case, let's just take a look at that selector. And
  8. if we do that, we can see all of the options right
  9. here in the HTML for the page. And based on
  10. our understanding of HTML, we know that it's these values here
  11. that we would need to submit as part of our post
  12. request. Alright, the options for airports look very similar. So instead
  13. of looking at that, let's take a look at The data
  14. that's here. Now, a couple things that I want to point out
  15. here, before we actually look at the HTML again. One is
  16. that for any given airport, this is reporting both domestic and
  17. international flights for that airport. Now in my case, I'm not actually
  18. interested in that distinction. I simply want to know arrivals and departures.
  19. So where does value in both columns, for a given month? I'm
  20. simply going to add them together. So this is one place that
  21. I'm doing a little bit of reshaping of the data. It's also
  22. the case that I've got these totals here in this columns, and
  23. then at the end of the rows for any given year. I'm
  24. simply going to ignore those. Again, a little bit of reshaping. Okay.
  25. So let's take a look at these elements. Now as you
  26. might expect, these are laid out in a table, and if
  27. I scroll to the top, I can actually see there's a
  28. class attribute for this particular table here. And this is going to help
  29. me when it comes time to actually parse this HTML in
  30. order to extract the data. So we've looked at, both how to
  31. go about getting the values we're going to need to submit
  32. in a post request in order to get the data we need.
  33. And then we've looked at once that data is presented to us, or
  34. in this case, to our program that's going to be accessing the site.
  35. How do we go about finding that data and pulling that data out
  36. of the HTML? Or at least where is it located in the HTML file?