YouTube

Got a YouTube account?

New: enable viewer-created translations and captions on your YouTube channel!

English subtitles

← User Agent Header - Web Development

Get Embed Code
2 Languages

Showing Revision 4 created 05/25/2016 by Udacity Robot.

  1. >> Okay, so I want to add a little more about user
  2. agents. So it's, it's one of the most important headers in
  3. an HTTP request. And when we were doing Reddit, user agents
  4. were really important to us. So we had the site that was
  5. online and really popular. And users were always you know, often
  6. writing scripts to, to pull content out on Reddit. And, you
  7. know, mostly they were doing good things. They were building you
  8. know, tools to, you know, do data collections so they can do
  9. some, you know, cool blog post about how Reddit works and that sort
  10. of thing. Sometimes they're doing bad things.
  11. You know, there are spammers, you know,
  12. looking for weaknesses or, you know, looking for ways to game the system.
  13. And user agents are important because sometimes, you know, a user would hit us
  14. a little too hard too fast, you know, basically, you know, hurting the
  15. website for real users. And if they had a legitimate user agent, we could
  16. look at them. We can you know, rate limit them accordingly or we can
  17. reach out to them and say, hey you know, you are hurting us and
  18. you know, they, they would adjust, and it's cool. The Googlebot is
  19. a really good example of this. The Googlebot is Google's WebCrawler that indexes
  20. website for, you know, their search engine. Googlebot would just punish us.
  21. They would hit us as fast as they could, and sometimes that would
  22. actually slow down the site. And until we learned how to handle
  23. that properly, we would just, you know, rate limit them a little bit.
  24. And, you know, Googlebot, you know, their feelings weren't hurt, they can
  25. still index the site. We just made them do it slower. But when
  26. people show up with fake user agents and we can tell that,
  27. you know, this one IP and this one user agent is just
  28. pummeling us and, and they're trying to pretend to be a browser,
  29. we would just block them altogether. So, using good user agents when you're
  30. writing software that, interacts with other people's websites is, is a really
  31. nice, courteous thing to do. And it's one of the things that
  32. makes the web work well for everybody. So it's always important to
  33. have a nice accurate user agent and to be honest when you can.
  34. >> Okay, now that I'd like you to do is promise
  35. to always use user agents appropriately when you're writing your web software.