For my first assignment I wanted to scrape comments from one of the articles on breitbart.com

Here’s the outcome:

CNN needs it's license pulled by FCC for Fraud! CNN is Anti-America Propaganda.Trump Draing The Swamps of MSM, PC Police, Left Wing Loons....
 
CNN is agitating and suggesting violence in some cases with their 'reporting' and manipulation of the stories they cover.Lying and subverting the truth are beyond Fake.Domestic Terrorism is what exactly ??
 
President Trump should implement a "3-strikes and you're out" rule for the news outlets. On the third blatant lie they get their FCC license pulled......CNN would be gone before dinnertime!!!
 
Simply allow the FCC to authorize "A Là Carte Cable"...Oh wait a minute! The new conservative head of the FCC--Ajit Pai--is working on THAT as we speak!#WINNING
 
just think how many garbage channels would go away, all those liberal entertainers out of work
 
It is impossible to make America great again without making America's MSM American again.6 anti-America globalist corporations should not control "truth" by controlling 95% of America's media.Trump and Trump's FCC will free America to become America again.
 
This guy illustrates your point beautifully. https://www.youtube.com/watch?...
 
I think protection of Political Incorrectness as part of freedom of speech will naturally burst the media's bubble.
 
the false outrage at Political Incorrectness is all part of the act. The Media and Trump are in a love hate relationship. Neither can function without the other
 
^1,000s^^^^^^^^^
 
Americans need to identify the anti-America owners of these MSM corporations/companies. Sulzberger Family for one. NYT & Company 
 
And Net Neutrality will stop the rest of the Leftist propaganda in its tracks. They rely on a free Internet. Net Neutrality will stop a free Internet and turn it into a proper capitalist marketplace. Then every liberal website in the entire world will evaporate within a month.
 
Start with that other O ... and her OWN network ... she makes me sick, Lena Dunham sick
 
Chaos !!That is this week's new word for the News to use, like 'Gravitas', and 'Temperament'. So listen for which of the FOX personalities work in the word CHAOS, and we will know who is on the new journ-O-list roster.
 
Shepturd Smith has added "CHAOS" to his Libturdal proto-vocabulary and is ululating it from his hysterical, LGBTQ perch at FoxSnooze...
 
Michael Eric Ricin is hyping the word ‘’heinous’’ which by no coincidence rhymes with chaos.
 
I loathe him and all his demeaning anti Trump statements during the election that we saw on YouTube . 
 
Exactly.Murdoch's whorehouse is peppered with varying degrees of globalist MSM presstitute.When Billy Bush tapes broke, even Hannity asked accusingly "Can Trump survive this", the MSM meme-de-jour.
 
Tomorrow’s word du jour will be verisimilitude.
 
CNN's STOCK WOULD CRATER OVERNIGHT...#NoMoreCableSubsidies#A_La_Carte_Cable_Now
 
I really LOVE this a la carte idea!!!! I cancelled cable a year ago but would like to watch my choice of programming. 
 
Sounds great. Libs will eventually lose their minds and may even move to Canada. 
 
about67 channels,watch about 6 of them.
 
Years ago I had 500 channels on dish and watched only 3 . Then I put up an antenna and get 35 better channels for free .
 
What??!!??With every single show available on the internet, a la carte, already, people still pay for full cable? Didnt know that.
 
There would certainly be more rioting, and even MORE people would get behind Trump!!
 
The only entertainer that should be allowed on TV is Dennis Miller. Now HE is truly funny. The rest of those 'comedian' talk show hosts should be put in prison.
 
leaving you with what? CSPAN and the military channel
 
Really !?? Damn that's good Pattie
 
15 zillion up votes.We watch less than 10 channels that we like, but subsidize a couple of hundred that we wouldn't watch on a BET.
 
BET, perhaps, being one of them. :-)
 
whaaaaaatspanish channelitalian channelno white channel?why do all the network series have blacks in positions of power ordering whites, often portrayed as stupid, around?what's new on mr. jello pudding
 
That would be a huge step forward but not enough to enlist me back into the ranks of TV viewers after 14 years without. One of the most interesting things about the advent of cable is that when the channel availability went from 3 and sometimes 4 in most markets to 20 or more, people still complained "there's nothing on TV tonight". As far as I'm concerned there seldom ever has been.
 
RE: authorize"A Là Carte Cable".Internet Streaming already is A Là Carte. There are no monthly charges like Cable TV and 90% of the channels are free. I pay nothing except for my Internet Connection but I need that anyway. The programming available on Internet Streaming is way better than anything on Cable TV. People invariably ask if they can get Cable TV Channels on a ROKU Box. I respond that after you switch you'll never want to watch those Cable TV Channels once you start watching the High Quality Programming available on the Internet. Cable TV has been a monopoly for so many decades and it's leftward bent so perverse you'll never miss those channels. That said you can pay $25 a month for 20 plus Cable TV channels via Sling.com but you'll quit subscribing to them after that first month. HBO is now available as a subscription channel without having an active Cable TV subscription. Lots of people continue paying for Cable TV just for FOX News. FOX News free live streaming is available on many Youtube channels so you can continue watching it without paying for Cable TV. There is no reason to keep paying for Cable TV when a ROKU Box provides thousands of high quality channels that are free or a fraction of the cost of that crap quality Cable TV Subscription.
 
That's what we did and its true!! After a month you don't miss it anymore !!!I didn't know about the HBO channels so thank you for the info. 
 
I would sure like to be able to watch my Cardinals again without supporting all that anti-American trash. Hardly worth it when you only watch 1 of hundreds of channels.
 
MLB has their At Bat app and their own MLB.TV. Watch your favorite team with an internet connection. No cable/satellite required. I use the At Bat app to listen to the Indians with their feed. Best part is that it can travel with you. Hope this helps.
 
We have that, it is not quite the same. Thanks for the reference though. What we have noticed is that once you give up cable and watching the games regularly you really don't miss them. I have read that MLB is cutting off its' nose to spite its' face, as their insistence on the cable packaging means that they are losing younger viewers in droves. Doesn't bode well.
 
Man I hope so there are station we should not pay for . or even want
 
This will be great All these channels we subsidize ..multiple Spanish channels,Bet channel.black starz,black hboAl jazzeer, etc,etc,etc I dont watch these stations..These channels arent for my viewing so why should i pay for it
 
THIS!!!! ^^^^^^^
 
Don't think Trump can do that! But I would like to see the White House revoke their Press Conferences passes.
 
YES!! Now that would be fun to watch...all the cnn aholes crying and whining about how mean Trump is....and how important the news is!! ha ha ha
 
or put them in the back of the room where they cant ask questions!
 
should be a separate room of the press briefing room so they can watch through the window while the other reporters ask questions - while they're at it, throw in a few stink bombs with them so they can feel really "at home"..
 
sorta like prison executions lol
 
They deserve prison and suspended sentences after convictions...https://www.merriam-webster.co...
 
Remember, we are talking about criminal acts, conspiracy and aiding and abetting, and cover up of crimes... Trump can and will likely make that a part of the "things he can do" if they, CNN and others, continue to commit criminal acts.Yep, that bad... support of a known felon to run for POTUS (hilduhbeast) is IN FACT a criminal actThere is a law called misprision of treason, misprision of felonyhttps://www.merriam-webster.co...
 
They are going to start using Skype to allow the media from communities 50 miles outside of DC to attend these press conferences - brilliant move to show that there are opinions outside the NY-DC crowd - that will reduce their influence - these Trump folks are SMART!
 
Trump is really using technology well. I love being able to bypass the dinosaur media to hear what my president has to say -- directly from him.

Since the site is heavy on JavaScript, scraping using beautifulsoup didn’t go that well. I tried using Selenium but it kept throwing error that it cannot connect to ChromeDriver, so I at the end I had to use JavaScript in the console:

Scraper:

 

I also had another scraper to scrape article titles and it worked but the results weren’t that interesting, so I chose not to proceed with it.

Questions:

  1. Can we go over Selenium in class?
  2. Saving outcome from command line using “>”: it saves only the first item on the list. How can we save all the items?

 

Leave a Reply

Your email address will not be published. Required fields are marked *