Oh, module 2. You finicky beast. See, module 2 is where the train derailed. I did exercise 1, the Dream Case, and then never really picked up things again.
I REGRET THIS. DEEPLY.
I don’t really know what else to say that I didn’t in my notes, so I’m going to just insert those here.
Hist3907B Module 2 Exercise 3: APIs
Well. This was… a thing.
To start, the prompt given in the Programming Historian’s instructions for installing HomeBrew didn’t work, which led to my panicking, and then further panicking, and research into whether I actually wanted to deal with sudo. Apparently I did.
I read through the instructions, I read through Steve Marti’s post “Canadiana in Context”; Ian Milligan’s original post on JSON; and attempted to learn how shell script works.
Going through all of these things, I thought I understood what was going on. I thought that all I needed to do to make everything work was to substitute my JSON URL in for the one given in the example. I couldnt’ imagine that given my illiteracy in all things related to coding, that I would be expected to change anything else. I continued to follow the instuctions.
And then I proceeded to enter the command given:
sudo chmod 700 file.sh … This is one of the biggest areas where I’ve had difficulty – understanding that some of the instructions are simply guidlines for what I actually need to input. So after all was said and done, and a lot of trial and error, I discovered that the only command that gave me something was:
sudo chmod api-canadiana-ottawa-search.sh.
Was this something what I wanted it to be? To be honest, I’m not really sure. I don’t think so. I’ve included a screen shot in my notebook of my terminal after entering the above command, as well as the one that was supposedly supposed to ‘run’ the program:
From the looks of things, it seems I might be close? I’ve quadruple checked to make sure that I have all the programs installed that I’m supposed to. I’m leaning towards the issue maybe being that I’ve goofed a bit on how to properly input my URL into where the example one was.
For good measure, this was the search URL I was working with: http://search.canadiana.ca/search?q=Ottawa&field=&df=1800&dt=1900&fmt=json
Aaaaaand then. I went back, re-read some things. Decided to play around some more. I may now be stuck in an endless cycle of downloading all 56,249 results from the non-JSON URL…
Aaaaand, we got to learnt he fun new command:
pkill -9 wget
Managed to stop the downloading of what apparently is not helpful data. Sorry Canadiana!
Followed the EXACT directions from archive.org
Result: “No URLs found in ../itemlist.txt”.
On an altered runthrough, in which I copied the list from the original, save as from .cvs file into a brand new text file, I got a scramble of stuff that ultimately ended in:
Excited by something different, I tried again, for science…
“No URLs found in ./itemlist.txt.”
I’m done for the day. I’ve been at this all evening. I can’t do the next module until I actually manage to ‘wrangle’ some data.
BUT WAIT, THERE’S MORE
Well, after yesterday’s epic fail and having a meltdown for most of the morning over how I am handing my assessment in late because, ya know, I didn’t do anything for the entirety of the course. We’re back.
I was mighty panicked, as I had no idea how I was supposed to figure out what I did wrong. I thought I’d gone through everything!
I looked at the JSON Response Document. I hoped it would tell me what I’d done wrong. Or at least give me some idea. No such luck.
I scrolled through the instructions again, including those for Windows. I found the conversation between Pickering and Dr. Graham. This finally did help me a bit. In previous attempts, I’d been inputting my own URL into line 28 of the code. Reading Dr. Graham’s explanation, I realized that it was only lines 9 and 20 that I needed to change.
Full of joy and hopes that I had not, in fact, completely screwed myself over; I looked through my notes from yesterday again, as well as looking through the instructions again.
I noticed that I had used the command
sudo chmod api-canandiana-ottawa-search.sh. This had yielded me a weird mishmash of information that didn’t really make sense. But the instructions say
sudo chmod **700** file.sh. I did try to search the command to see what this means, but ultimately, I have no idea, but crossed my fingers, prayed to the technology gods; and typed:
sudo chmod 700 api-ottawacanadiana.sh
I’ll admit, I screamed a little bit. And then shook my head at how overthinking things had led to me wasting hours on trying to figure things out.
And now I wait. For this to stop:
(Side note: It kind of never stopped. I think I forgot that it was still running after a certain point and accidentally closed my laptop. This was also when I realized that I was hooped, because these things take time, and I had none)
So, in summary. This was the exercise that shook me down to the core. It’s the one that made me throw my laptop down in frustration and pace around my room alternately questioning what I could have possibly done wrong and berating myself for not doing things before the deadline.
To be completely honest, I’m considering printing out the notes from this exercise and poster-ing them to my ceiling as a constant reminder to never leave things to such a thin wire again. This was the “you better buckle down, and do it fast, kid” moment. So I did. And even more than that, I learned that I am going to make mistakes. lots of them. So I needed to be prepared. I spent so much time (hours, I’m telling you, far more hours than ever should have been committed to such a thing) freaking out over something that was solved by calmly reading through the instructions and reviewing my own work. I learned how to use wget, expanded my knowledge of navigating via the command line, can now add the skill of “able to scrape data” to the resume… and yet what I learned was far more valuable: Things won’t always work. Leaving time to take a break so you can return to things with fresh eyes and a calm mentality is key.