A New Perspective: New major section
Moving along, though I'm spending way more time on getting up to speed than I originally expected. I suspect that I'm going to have to trim this a bit. All the more reason to get it versioned so I can reference old versions if I trim it too much. I fly out to BOS in under 13 days and my talk is in under 15. Getting close again. The difference this time, though, is that I don't have to do any research, which is saving me a ton of time compared to the last couple of years, which were research talks. Much less stressful. I could give the presentation ad-hoc without preparation, though it'd be a tad bit messy and disorganized if I did that. Alright, now I feel like I'm stalling in typing this commit message. Back to work for me.master
parent
be44088c95
commit
2fcfd61291
GPG Key ID:
8C917B7F5DC51BA2
|
|
@ -1,3 +1,5 @@
|
|||
# generated
|
||||
ff-*.png
|
||||
ff-edit-find.png
|
||||
ff-find-matches.png
|
||||
ff-find-menu.png
|
||||
|
||||
|
|
|
|||
Binary file not shown.
|
After Width: | Height: | Size: 61 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 128 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 26 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 66 KiB |
931
slides.org
931
slides.org
|
|
@ -21,7 +21,7 @@
|
|||
- GUIs /are/ sometimes the most efficient means of melding mind and machine.
|
||||
- De-emphasize doing everything on a terminal just because its "cool".
|
||||
|
||||
** Concepts [6/27]
|
||||
** Concepts [7/28]
|
||||
- [ ] =/sys= and such
|
||||
- [ ] Automating multi-step workflows with GNU Make
|
||||
- [ ] Concurrency
|
||||
|
|
@ -49,7 +49,7 @@
|
|||
- [ ] Evolve by making portions of command dynamic (variables, subshells)
|
||||
- [ ] Transfer commands from history into scripts and aliases for re-use
|
||||
- [ ] Interactive prototyping
|
||||
- [X] Looping ([[*Browser Topics][Browser Topics]])
|
||||
- [X] Looping ([[*Perspective Topics][Perspective Topics]])
|
||||
- [ ] McIlroy and Knuth
|
||||
- [X] Mouse has dependency on positioning of UI elements ([[*Browser Topics][Browser Topics]])
|
||||
- Changes over time, subject to user/system preferences, etc
|
||||
|
|
@ -58,18 +58,19 @@
|
|||
- [ ] Phone as an example of a device that best works with touch and GUIs
|
||||
and sacrifices practical freedom.
|
||||
- Requires specialized knowledge and is inconvenient to work with.
|
||||
- [X] Pipelines instead of intermediate files ([[*Browser Topics][Browser Topics]]).
|
||||
- [X] Pipelines instead of intermediate files ([[*Perspective Topics][Perspective Topics]]).
|
||||
- [ ] Practically exercising software freedom [0/2]
|
||||
- [ ] Users need to be able to convey their thoughts to the computer
|
||||
without being programmers
|
||||
- [ ] Concise primitives / building blocks
|
||||
- [ ] Readline (history and editing)
|
||||
- [X] Regular expressions ([[*Browser Topics][Browser Topics]])
|
||||
- [X] Regular expressions ([[*Perspective Topics][Perspective Topics]])
|
||||
- [ ] Remote commands via SSH
|
||||
- [X] Text as a universal interface
|
||||
- [X] All programs can work with one-another ([[*Browser Topics][Browser Topics]])
|
||||
- [X] All programs can work with one-another ([[*Perspective Topics][Perspective Topics]])
|
||||
- [X] Can replace any part of the process with a human---no difference
|
||||
between input from a program vs. from a keyboard. ([[*Browser Topics][Browser Topics]])
|
||||
- [X] The Unix Philosophy ([[*Perspective Topics][Perspective Topics]])
|
||||
- [X] Using keybindings in a GUI with no mouse ([[*Browser Topics][Browser Topics]])
|
||||
- Including transferring data between programs
|
||||
- [ ] Using macros to script keypresses (Vim)
|
||||
|
|
@ -110,7 +111,7 @@
|
|||
- [ ] =vim=
|
||||
- [ ] =screen=
|
||||
|
||||
* DEVOID Slides [0/4]
|
||||
* DEVOID Slides [0/5]
|
||||
:PROPERTIES:
|
||||
:ID: slides
|
||||
:END:
|
||||
|
|
@ -129,7 +130,7 @@
|
|||
|---------------------------------------------------+----------+--------+-------------|
|
||||
| \_ Practical Freedom | | DEVOID | fullframe |
|
||||
|---------------------------------------------------+----------+--------+-------------|
|
||||
| \_ Practical Example: Web Browser | 1:00 | DEVOID | |
|
||||
| \_ Practical Example: Web Browser | 1:00 | RAW | |
|
||||
| \_ Browser Topics | | | |
|
||||
| \_ Example: Web Browser | 1:00 | RAW | frame |
|
||||
| \_ Finding Text (Mouse-Driven GUI Interaction) | | RAW | frame |
|
||||
|
|
@ -141,6 +142,25 @@
|
|||
| \_ GUIs of a Feather | | RAW | fullframe |
|
||||
| \_ Macro-Like Keyboard Instructions | | RAW | fullframe |
|
||||
|---------------------------------------------------+----------+--------+-------------|
|
||||
| \_ A New Perspective | | DEVOID | |
|
||||
| \_ Perspective Topics | | | |
|
||||
| \_ Secrets? | | RAW | fullframe |
|
||||
| \_ Lifting the Curtain | | RAW | frame |
|
||||
| \_ Web Page Source Code | | RAW | block |
|
||||
| \_ Text | | RAW | fullframe |
|
||||
| \_ Text is a Universal Interface | | RAW | fullframe |
|
||||
| \_ The Shell Command Prompt | | RAW | frame |
|
||||
| \_ Eliminating the Web Browser | | RAW | frame |
|
||||
| \_ Browser vs. =wget= Comparison | | RAW | frame |
|
||||
| \_ Finding Text on the Command Line | | RAW | frame |
|
||||
| \_ A More Gentle Reply | | RAW | frame |
|
||||
| \_ Writing to Files (Redirection) | | RAW | frame |
|
||||
| \_ Starting Our List | | RAW | fullframe |
|
||||
| \_ Command Refactoring | | RAW | fullframe |
|
||||
| \_ Again: Text is a Universal Interface | | RAW | againframe |
|
||||
| \_ Pipelines | | RAW | fullframe |
|
||||
| \_ Summary of the Unix Philosophy | | RAW | fullframe |
|
||||
|---------------------------------------------------+----------+--------+-------------|
|
||||
| \_ Thank You | 00:00:01 | | fullframe |
|
||||
#+END:
|
||||
|
||||
|
|
@ -196,7 +216,7 @@ Practical Freedom
|
|||
|
||||
|
||||
** RAW Practical Example: Web Browser [0/9]
|
||||
*** Browser Topics [3/8] :noexport:
|
||||
*** Browser Topics [3/3] :noexport:
|
||||
This example is the main segue from GUIs into the utility of keybindings
|
||||
into shell, so it has to be compelling. I chose something that is
|
||||
frequently done by users: visiting webpages and searching for text.
|
||||
|
|
@ -234,39 +254,6 @@ frequently done by users: visiting webpages and searching for text.
|
|||
- [X] This would work with nearly /any/ combination of web browser and
|
||||
text editor.
|
||||
- [X] But what if I had 10? 100? 1000? This is still tedious and slow.
|
||||
- [ ] What if I could walk away and get a coffee, play with the kids,
|
||||
come back and have it done for me?
|
||||
- Emphasize how users familiar with the tools I haven't yet mentioned
|
||||
may know how to do this, but people who don't know of Unix tools
|
||||
will still be thinking in terms of the UI, trying to figure out how
|
||||
to automate it.
|
||||
- [ ] Right-click on page and View Source
|
||||
- Briefly mention HTML and =C-f=, show how the text is on the page.
|
||||
- [ ] Open a terminal [0/4]
|
||||
- [ ] Type =wget URL=
|
||||
- [ ] Show how it downloaded a file, useful for offline viewing
|
||||
- [ ] Open it in a text editor, show how the text is there. =C-f=.
|
||||
- We eliminated the web browser and can still find the text
|
||||
/using the same keybinding/.
|
||||
- [ ] =grep word file.html=
|
||||
- We have just eliminated the editor.
|
||||
- [ ] Eliminate the temporary file entirely using a pipe
|
||||
- Show both =wget -O-= and =curl=.
|
||||
- [ ] Now script discovering what pages contain a certain word [0/4]
|
||||
- [ ] Mention previous example of being emailed a list of URLs. Rather
|
||||
than pasting them into a file, let's discover them using the same
|
||||
tool: =grep=. Save email to a file.
|
||||
- [ ] =grep -o 'https\?://[^ ]\+'=, show how it gets a list of URLs.
|
||||
- Mention how powerful regexes are, e.g. "find all the phone numbers on
|
||||
a page".
|
||||
- g/re/p
|
||||
- [ ] Introduce =while= and =read=, showing how we can iteratively run
|
||||
commands, maybe =... | while read url; do echo "URL: $url"; done=.
|
||||
- [ ] Introduce =-q= flag for grep, and use it to conditionally output
|
||||
only the URLs at which the word can be found.
|
||||
- Not enough talk time to walk away and get a coffee, but let's see if
|
||||
we can maybe let it run for 10--30s while I blabber on. Depends on
|
||||
the connection speed at MIT with all the participants.
|
||||
|
||||
|
||||
*** RAW Example: Web Browser :B_frame:
|
||||
|
|
@ -807,6 +794,868 @@ And that's why I've dragged you through this drawn-out example---to
|
|||
enhancements to our workflow.
|
||||
|
||||
|
||||
** DEVOID A New Perspective [0/16]
|
||||
*** Perspective Topics [12/13] :noexport:
|
||||
- [X] What if I could walk away and get a coffee, play with the kids,
|
||||
come back and have it done for me?
|
||||
- Emphasize how users familiar with the tools I haven't yet mentioned
|
||||
may know how to do this, but people who don't know of Unix tools
|
||||
will still be thinking in terms of the UI, trying to figure out how
|
||||
to automate it.
|
||||
- [X] Right-click on page and inspect an element.
|
||||
- Briefly mention DOM and how this represents the current state of
|
||||
the page.
|
||||
- Maybe show how it can be modified to illustrate the structure better.
|
||||
- But I don't want to get too deep into this.
|
||||
- [X] Right-click on page and View Source.
|
||||
- Explain how this is like the source code to the webpage (HTML).
|
||||
- [X] Copy text into editor, =C-f=.
|
||||
- We can still find the text /using the same keybinding/.
|
||||
- [X] Save HTML file.
|
||||
- [X] Demonstrate opening it in a web browser vs. an editor.
|
||||
- Same exact document, different ways of rendering it.
|
||||
- [X] Open a terminal [4/4]
|
||||
- [X] Type =wget URL=
|
||||
- [X] Show how it downloaded a file
|
||||
- [X] Open it in a text editor, show how the text is there. =C-f=.
|
||||
- We eliminated the web browser
|
||||
- [X] =grep word file.html=
|
||||
- We have just eliminated =Ctrl+F= in the editor.
|
||||
- [X] But it's too verbose, so show =grep -q && echo=.
|
||||
- Talk about echo and exit status briefly, but don't go into exit codes.
|
||||
- [X] The goal is to output a URL to a file
|
||||
- Introduce redirection (overwrite and append)
|
||||
- [X] Now we have duplication: URL in two places
|
||||
- Introduce variables
|
||||
- [X] Eliminate the temporary file entirely using a pipe
|
||||
- [X] Introduce the Unix philosophy
|
||||
- [ ] Now script discovering what pages contain a certain word [0/4]
|
||||
- [ ] Mention previous example of being emailed a list of URLs. Rather
|
||||
than pasting them into a file, let's discover them using the same
|
||||
tool: =grep=. Save email to a file.
|
||||
- [ ] =grep -o 'https\?://[^ ]\+'=, show how it gets a list of URLs.
|
||||
- Mention how powerful regexes are, e.g. "find all the phone numbers on
|
||||
a page".
|
||||
- g/re/p
|
||||
- [ ] Introduce =while= and =read=, showing how we can iteratively run
|
||||
commands, maybe =... | while read url; do echo "URL: $url"; done=.
|
||||
- [ ] Introduce =-q= flag for grep, and use it to conditionally output
|
||||
only the URLs at which the word can be found.
|
||||
- Not enough talk time to walk away and get a coffee, but let's see if
|
||||
we can maybe let it run for 10--30s while I blabber on. Depends on
|
||||
the connection speed at MIT with all the participants.
|
||||
|
||||
*** RAW Secrets? :B_fullframe:
|
||||
:PROPERTIES:
|
||||
:BEAMER_env: fullframe
|
||||
:END:
|
||||
|
||||
# Slide intentionally left blank
|
||||
|
||||
**** Notes :B_noteNH:
|
||||
:PROPERTIES:
|
||||
:BEAMER_env: noteNH
|
||||
:END:
|
||||
|
||||
So what if I told you that,
|
||||
while the average user is cursing me out for sending them 1000 URLs,
|
||||
I could go grab some coffee and play with my kids and come back however
|
||||
much time later to a list that has been generated for me,
|
||||
and it'd still be done before the user has even had a chance to open
|
||||
all of the URLs,
|
||||
letalone check them?
|
||||
And what if I told you that it'd only take a minute or two to for me to
|
||||
create this process?
|
||||
|
||||
This is where the whole concept of ``wizardry'' comes in.
|
||||
Some of you are sitting in the audience or watching this remotely rolling
|
||||
your eyes thinking ``oh this guy thinks he's so sweet'',
|
||||
because the answer is obvious to you.
|
||||
But to those of you who are confined to the toolset that I just
|
||||
demonstrated...it's
|
||||
not going to be obvious.
|
||||
You may still be thinking in terms of that toolset---thinking
|
||||
of how we can continue to use those same tools.
|
||||
You don't know what you don't know.
|
||||
|
||||
The problem is that there is a whole world and way of computing that is
|
||||
hidden from most users.
|
||||
And it's not hidden because it's a secret.
|
||||
I'd hate to disappoint you,
|
||||
but there's no magic or secrets in this talk.
|
||||
It's because modern interfaces have come to completely mask it or provide
|
||||
alternatives to it that happen to be ``good enough'' for the job.
|
||||
|
||||
But ``good enough'' is only good enough until it's not.
|
||||
|
||||
*** RAW Lifting the Curtain :B_frame:
|
||||
:PROPERTIES:
|
||||
:BEAMER_env: frame
|
||||
:END:
|
||||
|
||||
#+BEAMER: \only<1>{
|
||||
#+ATTR_LATEX: :width 2in
|
||||
[[file:images/ff-inspect-menu.png]]
|
||||
#+BEAMER: }
|
||||
|
||||
#+BEAMER: \only<2>{
|
||||
[[file:images/ff-inspect.png]]
|
||||
#+BEAMER: }
|
||||
|
||||
# ATTR_LATEX does not appear to work with :trim or :clip
|
||||
#+BEAMER: \only<3>{\begin{center}
|
||||
#+BEAMER: \includegraphics[width=0.9\linewidth,trim={0px 0px 350px 375px},clip]{images/ff-inspect.png}
|
||||
#+BEAMER: \end{center}}
|
||||
|
||||
**** Notes :B_noteNH:
|
||||
:PROPERTIES:
|
||||
:BEAMER_env: noteNH
|
||||
:END:
|
||||
|
||||
Let's lift the curtain, so to speak, on what's really going on in the web
|
||||
browser.
|
||||
Don't worry,
|
||||
we're only going to give it a little peek;
|
||||
nothing too complicated.
|
||||
|
||||
Take the LP2019 speaker list page for instance.
|
||||
If you right-click on my name and click on ``Inspect Element'',
|
||||
you are presented with the developer tools for this browser which shows
|
||||
what makes up the webpage.
|
||||
You'll notice that it has a hierarchical structure made up of nodes
|
||||
describing how certain data ought to be formatted.
|
||||
For example,
|
||||
my name is in an =h2= element,
|
||||
which represents a heading.
|
||||
Below that we see the talk title and then the talk abstract in a paragraph
|
||||
element,
|
||||
denoted by =p=.
|
||||
|
||||
If we look in that bottom =p= element,
|
||||
you'll see the exact text that's displayed on the webpage.
|
||||
And sure enough,
|
||||
that text contains the term ``free software''.
|
||||
|
||||
Now,
|
||||
technically,
|
||||
this inspector represents something called the DOM,
|
||||
which represents the current state of the page.
|
||||
If it were dynamic,
|
||||
you'd see this updating.
|
||||
We could even change it in here and the page would update.
|
||||
But you don't need to use a specialized tool to view the structure of the
|
||||
initial webpage;
|
||||
I just did that for visualization,
|
||||
since it conveniently highlights the associated elements on the page
|
||||
which is useful for demonstration purposes.
|
||||
|
||||
*** RAW Web Page Source Code :B_block:
|
||||
:PROPERTIES:
|
||||
:BEAMER_env: block
|
||||
:END:
|
||||
|
||||
**** Columns :B_columns:
|
||||
:PROPERTIES:
|
||||
:BEAMER_env: columns
|
||||
:END:
|
||||
|
||||
***** Left :B_column:
|
||||
:PROPERTIES:
|
||||
:BEAMER_env: column
|
||||
:BEAMER_col: 0.30
|
||||
:END:
|
||||
|
||||
[[file:images/ff-view-source-menu.png]]
|
||||
|
||||
***** Right :B_column:
|
||||
:PROPERTIES:
|
||||
:BEAMER_env: column
|
||||
:BEAMER_col: 0.70
|
||||
:END:
|
||||
|
||||
[[file:images/ff-view-source.png]]
|
||||
|
||||
**** Notes :B_noteNH:
|
||||
:PROPERTIES:
|
||||
:BEAMER_env: noteNH
|
||||
:END:
|
||||
|
||||
If we instead select "View Page Source" from the context menu,
|
||||
then we get a new tab that contains a much uglier version of what we just
|
||||
saw in the DOM inspector.
|
||||
This is the raw source code of the webpage.
|
||||
Most of it, anyway.
|
||||
It is a document language called HTML.
|
||||
And as you may have noticed,
|
||||
it's plain text.
|
||||
Structured, but plain, text.
|
||||
|
||||
And as you can see,
|
||||
``free software'' is there all the same.
|
||||
We don't need to view the webpage with all its fancy formatting.
|
||||
For the problem we're trying to solve,
|
||||
it provides little benefit.
|
||||
|
||||
*** RAW Text :B_fullframe:
|
||||
:PROPERTIES:
|
||||
:BEAMER_env: fullframe
|
||||
:END:
|
||||
|
||||
#+BEAMER: \fullslidetext
|
||||
Text.
|
||||
|
||||
**** Notes :B_noteNH:
|
||||
:PROPERTIES:
|
||||
:BEAMER_env: noteNH
|
||||
:END:
|
||||
|
||||
As we're about to see,
|
||||
this simple fact---that
|
||||
that the webpage is represented by plain text---opens
|
||||
up a whole new world to us.
|
||||
We have stripped away all the complex visual GUI stuff and we're left with
|
||||
the raw substance of the page which still contains the information that we
|
||||
are looking for.
|
||||
|
||||
But we're still within the web browser.
|
||||
|
||||
We don't have to be.
|
||||
We can copy all of that text and paste it into our editor.
|
||||
=Ctrl+A Ctrl+C Alt-Tab Ctrl+P=.
|
||||
And sure enough,
|
||||
search works all the same.
|
||||
=Ctrl+F= and we can still find ``free software''.
|
||||
Completely different program,
|
||||
and we can still find the text using the same keybinding.
|
||||
|
||||
*** RAW Text is a Universal Interface :B_fullframe:
|
||||
:PROPERTIES:
|
||||
:BEAMER_env: fullframe
|
||||
:END:
|
||||
|
||||
#+BEAMER: \fullslidetext
|
||||
Text is a Universal Interface
|
||||
|
||||
**** Notes :B_noteNH:
|
||||
:PROPERTIES:
|
||||
:BEAMER_env: noteNH
|
||||
:END:
|
||||
|
||||
Text is a universal interface.
|
||||
And what I mean by that is---you
|
||||
don't need any special tools to work with it.
|
||||
You can view it in your web browser.
|
||||
You can view it in your text editor.
|
||||
You can paste it in a text message.
|
||||
You can put it in a book.
|
||||
You can write it down on a paper and type it back into your computer.
|
||||
|
||||
Text is how we communicate with one-another as human beings.
|
||||
|
||||
But the simplicity of text has practical computational benefits, too.
|
||||
Let me show you.
|
||||
|
||||
Let's save this HTML as a file,
|
||||
=speakers.html=.
|
||||
|
||||
If we opened this file,
|
||||
it would open in our web browser and we would see the same webpage,
|
||||
although it would look a bit different since a lot of the styling is
|
||||
stored outside of this HTML file,
|
||||
and there won't be any images.
|
||||
But this isn't a talk about web development so I'm not going to go deeper
|
||||
than that.
|
||||
|
||||
But if again we opened this HTML file in our text editor,
|
||||
you would see that same plain text HTML as before;
|
||||
one just chooses to render it differently than another.
|
||||
|
||||
Even though we can view the HTML in our text editor,
|
||||
we haven't eliminated the web browser yet;
|
||||
we still need it to navigate to the webpage and view its source.
|
||||
But if that's all we're using the web browser for,
|
||||
then it's one hell of an inefficient way of telling the computer that we
|
||||
just want the HTML document at a certain URL.
|
||||
|
||||
Up until this point,
|
||||
the keyboard has been used as a secondary interface---as
|
||||
an /alternative/ to something.
|
||||
Now we're going to venture into a world where it is /the/ interface.
|
||||
|
||||
*** RAW The Shell Command Prompt :B_frame:
|
||||
:PROPERTIES:
|
||||
:BEAMER_env: frame
|
||||
:END:
|
||||
|
||||
#+BEGIN_SRC sh
|
||||
mikegerwitz@lp2019-laptop:~$
|
||||
# ^ user ^ host ^ working directory (home)
|
||||
#+END_SRC
|
||||
|
||||
#+BEAMER: \begin{uncoverenv}<2->
|
||||
#+BEAMER: \subskip
|
||||
This presentation will show:
|
||||
#+BEAMER: \medskip
|
||||
|
||||
#+BEGIN_SRC
|
||||
$ command
|
||||
output line 1
|
||||
output line 2
|
||||
...
|
||||
output line N
|
||||
#+END_SRC
|
||||
#+BEAMER: \end{uncoverenv}
|
||||
|
||||
**** Notes :B_noteNH:
|
||||
:PROPERTIES:
|
||||
:BEAMER_env: noteNH
|
||||
:END:
|
||||
|
||||
If you open a terminal,
|
||||
also called a VTE, or virtual terminal emulator,
|
||||
you will be greeted with a curious string of characters.
|
||||
This is a /command prompt/.
|
||||
|
||||
The program that is prompting you for a command is called the /shell/.
|
||||
The GNU shell is =bash=,
|
||||
which is the default on most GNU/Linux systems.
|
||||
It's also the default on Mac OSX,
|
||||
if you happen to be using that.
|
||||
And Windows now has something they call
|
||||
``Bash on Ubuntu on Windows'',
|
||||
which is GNU/Linux running atop of the Windows kernel.
|
||||
|
||||
Bash isn't required to run any of the commands I'm presenting as part of
|
||||
this talk,
|
||||
but I will be mentioning some features specific to bash,
|
||||
which I'll note when I do in case you happen to be using a different
|
||||
shell.
|
||||
|
||||
*** RAW Eliminating the Web Browser :B_frame:
|
||||
:PROPERTIES:
|
||||
:BEAMER_env: frame
|
||||
:END:
|
||||
|
||||
#+BEGIN_SRC sh
|
||||
$ wget https://libreplanet.org/2019/speakers/
|
||||
#+END_SRC
|
||||
|
||||
#+BEAMER: \begin{uncoverenv}<2>
|
||||
#+BEGIN_SRC sh
|
||||
--2019-03-24 00:00:00-- https://libreplanet.org/2019/speakers/
|
||||
Resolving libreplanet.org (libreplanet.org)... 209.51.188.248
|
||||
Connecting to libreplanet.org (libreplanet.org)|209.51.188.248|:443... connected.
|
||||
HTTP request sent, awaiting response... 200 OK
|
||||
Length: unspecified [text/html]
|
||||
Saving to: ‘index.html’
|
||||
...
|
||||
2019-03-24 00:00:00 (1.78 MB/s) - ‘index.html’ saved [67789]
|
||||
#+END_SRC
|
||||
#+BEAMER: \end{uncoverenv}
|
||||
|
||||
#+BEAMER: \begin{uncoverenv}<3>
|
||||
#+BEGIN_SRC sh
|
||||
$ wget -O speakers.html \
|
||||
https://libreplanet.org/2019/speakers/
|
||||
#+END_SRC
|
||||
#+BEAMER: \end{uncoverenv}
|
||||
|
||||
**** Notes :B_noteNH:
|
||||
:PROPERTIES:
|
||||
:BEAMER_env: noteNH
|
||||
:END:
|
||||
|
||||
The goal is to retrieve the HTML file at a given URL.
|
||||
GNU/Linux distributions usually come with GNU =wget=,
|
||||
which does precisely that.
|
||||
To invoke it,
|
||||
we type the name of the command,
|
||||
followed by a space,
|
||||
followed by the URL we wish to retrieve,
|
||||
and then hit enter.
|
||||
|
||||
What follows is quite a bit of text.
|
||||
The details aren't particularly important as long as it's successful,
|
||||
but notice that it says it saved to =index.html=.
|
||||
That's not intuitive to those who don't understand that name was used.
|
||||
|
||||
So let's tell =wget= what file we want to output to.
|
||||
We do this with the =O= option,
|
||||
like so.
|
||||
It takes a single argument,
|
||||
which is the name of the output file.
|
||||
The backslash here allows us to continue the command onto the next line;
|
||||
otherwise, a newline tells the shell to execute the command.
|
||||
|
||||
So remember previously that we manually created =speakers.html= by viewing
|
||||
the source of the webpage in Icecat.
|
||||
If we open this file,
|
||||
we'll find that it contains /exactly the same text/,
|
||||
and we never had to open a web browser.
|
||||
And we can search it all the same as before.
|
||||
|
||||
*** RAW Browser vs. =wget= Comparison :B_frame:
|
||||
:PROPERTIES:
|
||||
:BEAMER_env: frame
|
||||
:END:
|
||||
|
||||
#+BEGIN_SRC
|
||||
Ctrl+L ``https://libreplanet.org/2019/speakers/''
|
||||
#+END_SRC
|
||||
|
||||
#+BEAMER: \subskip
|
||||
|
||||
#+BEGIN_SRC sh
|
||||
$ wget https://libreplanet.org/2019/speakers/
|
||||
#+END_SRC
|
||||
|
||||
**** Notes :B_noteNH:
|
||||
:PROPERTIES:
|
||||
:BEAMER_env: noteNH
|
||||
:END:
|
||||
|
||||
This is a very different means of interacting with the computer,
|
||||
but if we compare this with the keyboard shortcut used previously,
|
||||
they are very similar.
|
||||
Not so scary,
|
||||
right?
|
||||
It's hard to imagine a more direct line of communication with the computer
|
||||
for downloading a webpage,
|
||||
short of reading your mind.
|
||||
|
||||
If we then try to output to a specific file,
|
||||
then it's even /easier/ on the command line.
|
||||
It's true that you can save the HTML using Icecat by hitting =Ctrl+S=,
|
||||
but that saves a lot more than just the HTML page---it
|
||||
also saves all the images and stylesheets and other resources,
|
||||
which is much more than we need.
|
||||
|
||||
*** RAW Finding Text on the Command Line :B_frame:
|
||||
:PROPERTIES:
|
||||
:BEAMER_env: frame
|
||||
:END:
|
||||
|
||||
#+BEGIN_SRC sh
|
||||
$ grep 'free software' speakers.html
|
||||
#+END_SRC
|
||||
|
||||
#+BEAMER: \begin{uncoverenv}<2->
|
||||
\vdots
|
||||
#+BEGIN_SRC
|
||||
<p>Mike Gerwitz is a free software hacker and activist with a focus on
|
||||
exclusively free software. Mike spends most of his free time with his
|
||||
#+END_SRC
|
||||
\vdots
|
||||
#+BEAMER: \end{uncoverenv}
|
||||
|
||||
**** Notes :B_noteNH:
|
||||
:PROPERTIES:
|
||||
:BEAMER_env: noteNH
|
||||
:END:
|
||||
|
||||
Not having to open a web browser is nice,
|
||||
but having to run =wget= and then open the downloaded HTML file is a bit
|
||||
of a pain;
|
||||
is there a command that can help us there too?
|
||||
|
||||
We want to know whether a page contains the term ``free software''.
|
||||
For that we use a tool called =grep=.
|
||||
|
||||
The first argument to =grep= is the search string,
|
||||
and the remaining arguments---just one here---tell it where it should
|
||||
search.
|
||||
The first argument to =grep= is quoted because it contains a space,
|
||||
otherwise the shell would think our search phrase was only `free' and that
|
||||
the files we wanted to search were `software' and `speakers.html'.
|
||||
|
||||
You'll get a bunch of output;
|
||||
I just included a small snippet here.
|
||||
But notice how it happens to include exactly the text we were looking at in
|
||||
the web browser.
|
||||
|
||||
And with that we have replicated =Ctrl+F=.
|
||||
But did we do a good job conveying our thoughts to the machine?
|
||||
|
||||
We just wanted to know whether the page con
|
||||
|
||||
And with that we have replicated =Ctrl+F=.
|
||||
But did we do a good job conveying our thoughts to the machine?
|
||||
|
||||
We just wanted to know whether the page /contains/ the phrase;
|
||||
we don't care to see it!
|
||||
So while we have efficiently conveyed a search string,
|
||||
we didn't receive an efficient reply;
|
||||
it's information overload.
|
||||
|
||||
*** RAW A More Gentle Reply :B_frame:
|
||||
:PROPERTIES:
|
||||
:BEAMER_env: frame
|
||||
:END:
|
||||
|
||||
#+BEGIN_SRC sh
|
||||
$ grep --quiet 'free software' speakers.html && echo yes
|
||||
|
||||
yes
|
||||
#+END_SRC
|
||||
|
||||
#+BEAMER: \subskip
|
||||
|
||||
#+BEAMER: \begin{onlyenv}<2>
|
||||
#+BEGIN_SRC sh
|
||||
$ echo 'Hello, world!'
|
||||
|
||||
Hello, world!
|
||||
#+END_SRC
|
||||
#+BEAMER: \end{onlyenv}
|
||||
|
||||
#+BEAMER: \begin{onlyenv}<3>
|
||||
#+BEGIN_SRC sh
|
||||
$ grep --quiet 'open source' speakers.html || echo no
|
||||
|
||||
no
|
||||
#+END_SRC
|
||||
#+BEAMER: \end{onlyenv}
|
||||
|
||||
**** Notes :B_noteNH:
|
||||
:PROPERTIES:
|
||||
:BEAMER_env: noteNH
|
||||
:END:
|
||||
|
||||
First we tell =grep= to modify its behavior with the =q= flag,
|
||||
which stands for ``quiet''.
|
||||
Rather than outputting results,
|
||||
=grep= will exist silently;
|
||||
it will instead return a status to the shell that says whether or not
|
||||
the search failed.
|
||||
|
||||
POSIX-like shells,
|
||||
like Bash,
|
||||
offer the ability to say ``run this command if the previous succeeds'',
|
||||
and this is done by putting two ampersands between the commands.
|
||||
|
||||
The command to run if =grep= succeeds in finding a match is =echo=.
|
||||
All echo does is takes its arguments and spits them right back out again as
|
||||
output.
|
||||
So this essentially states:
|
||||
``search for `free software' in =speakers.html= and output `yes' if it is
|
||||
found''.
|
||||
|
||||
Since =echo= is its own command,
|
||||
it also works by itself.
|
||||
Here's the classic ``hello, world'' program in shell.
|
||||
|
||||
But if you recall our research task,
|
||||
it was to search for pages that do /not/ contain the term ``free
|
||||
software''.
|
||||
We can do that too,
|
||||
by using two pipes in place of two ampersands,
|
||||
which states:
|
||||
``search for `free software' in =speakers.html= and output `no' if it
|
||||
is /not/ found''.
|
||||
|
||||
*** RAW Writing to Files (Redirection) :B_frame:
|
||||
:PROPERTIES:
|
||||
:BEAMER_env: frame
|
||||
:END:
|
||||
|
||||
- Commands write to standard out (stdout) by default
|
||||
- /Output redirection/ writes somewhere else
|
||||
|
||||
#+BEAMER: \subskip
|
||||
|
||||
#+BEGIN_SRC sh
|
||||
# overwrites each time
|
||||
$ echo 'Hello, world!' > hello.txt
|
||||
$ echo 'Hello again, world!' > hello.txt
|
||||
|
||||
# appends (echo adds a newline)
|
||||
$ echo 'First line' >> results.txt
|
||||
$ echo 'Second line' >> results.txt
|
||||
|
||||
# truncates file (empties)
|
||||
> results.txt
|
||||
#+END_SRC
|
||||
|
||||
**** Notes :B_noteNH:
|
||||
:PROPERTIES:
|
||||
:BEAMER_env: noteNH
|
||||
:END:
|
||||
|
||||
Alright, we're well on our way now!
|
||||
But we still haven't gotten rid of that damn text editor,
|
||||
because we need to save a list of URLs to a file to hold our final
|
||||
results!
|
||||
|
||||
Well as it so happens,
|
||||
writing to a file is such a common operation that it's built right into
|
||||
the shell.
|
||||
We use a feature called /redirection/.
|
||||
|
||||
There are two types of output redirection.
|
||||
If you place a single greater-than symbol followed by a filename after a
|
||||
command,
|
||||
then the output of that command will replace anything already in the
|
||||
file.
|
||||
So the result of the first two command will be a =hello.txt= that contains
|
||||
only a single line:
|
||||
``Hello again, world!''.
|
||||
|
||||
The second type,
|
||||
which uses /two/ greater-than symbols,
|
||||
appends to the file.
|
||||
=echo= by default adds a newline,
|
||||
so the result of the second two commands is a =results.txt= containing two
|
||||
lines,
|
||||
``First line'' and ``Second line'' respectively.
|
||||
If the file doesn't yet exist,
|
||||
an empty one will be created before writing.
|
||||
|
||||
I think maybe you can see where I'm going with this.
|
||||
|
||||
*** RAW Starting Our List :B_fullframe:
|
||||
:PROPERTIES:
|
||||
:BEAMER_env: fullframe
|
||||
:END:
|
||||
|
||||
#+BEGIN_SRC sh
|
||||
|
||||
$ wget --quiet -O speakers.html \
|
||||
https://libreplanet.org/2019/speakers/ \
|
||||
&& grep --quiet 'free software' speakers.html \
|
||||
|| echo https://libreplanet.org/2019/speakers/ \
|
||||
>> results.txt
|
||||
#+END_SRC
|
||||
|
||||
**** Notes :B_noteNH:
|
||||
:PROPERTIES:
|
||||
:BEAMER_env: noteNH
|
||||
:END:
|
||||
|
||||
Take a look at that for a moment.
|
||||
/<pause ~5s>/
|
||||
Can anyone tell me what the result of this command line will be?
|
||||
/<pause ~5s>/
|
||||
/<react appropriately>/
|
||||
|
||||
As exciting as it is to start to bring these things together,
|
||||
the result is pretty anti-climatic---we
|
||||
know that =speakers.html= /does/ contain the string ``free software'',
|
||||
and so the result is that =results.txt= contains /nothing/!
|
||||
In fact,
|
||||
if =results.txt= didn't exist yet,
|
||||
it still wouldn't even exist.
|
||||
|
||||
/<for the sake of demonstration, maybe modify it to output>/
|
||||
|
||||
At this point,
|
||||
we have successfully eliminated both the web browser and text editor.
|
||||
But this is a hefty command to have to modify each time we want to try a
|
||||
different URL.
|
||||
|
||||
*** RAW Command Refactoring :B_fullframe:
|
||||
:PROPERTIES:
|
||||
:BEAMER_env: fullframe
|
||||
:END:
|
||||
|
||||
#+BEAMER: \begin{onlyenv}<5>
|
||||
#+BEGIN_SRC sh
|
||||
|
||||
$ wget --quiet -O speakers.html \
|
||||
https://libreplanet.org/2019/speakers/ \
|
||||
&& grep --quiet 'free software' speakers.html \
|
||||
|| echo https://libreplanet.org/2019/speakers/ \
|
||||
>> results.txt
|
||||
#+END_SRC
|
||||
#+BEAMER: \end{onlyenv}
|
||||
|
||||
#+BEAMER: \begin{onlyenv}<1>
|
||||
#+BEGIN_SRC sh
|
||||
$ URL=https://libreplanet.org/2019/speakers/
|
||||
$ wget --quiet -O speakers.html \
|
||||
"$URL" \
|
||||
&& grep --quiet 'free software' speakers.html \
|
||||
|| echo "$URL" \
|
||||
>> results.txt
|
||||
#+END_SRC
|
||||
#+BEAMER: \end{onlyenv}
|
||||
|
||||
#+BEAMER: \begin{onlyenv}<2>
|
||||
#+BEGIN_SRC sh
|
||||
$ URL=https://libreplanet.org/2019/speakers/
|
||||
$ wget -qO speakers.html \
|
||||
"$URL" \
|
||||
&& grep -q 'free software' speakers.html \
|
||||
|| echo "$URL" \
|
||||
>> results.txt
|
||||
#+END_SRC
|
||||
#+BEAMER: \end{onlyenv}
|
||||
|
||||
#+BEAMER: \begin{onlyenv}<3>
|
||||
#+BEGIN_SRC sh
|
||||
$ URL=https://libreplanet.org/2019/speakers/
|
||||
$ wget -qO - \
|
||||
"$URL" \
|
||||
| grep -q 'free software' \
|
||||
|| echo "$URL" \
|
||||
>> results.txt
|
||||
#+END_SRC
|
||||
#+BEAMER: \end{onlyenv}
|
||||
|
||||
#+BEAMER: \begin{onlyenv}<4->
|
||||
#+BEGIN_SRC sh
|
||||
$ URL=https://libreplanet.org/2019/speakers/
|
||||
$ wget -qO - "$URL" \
|
||||
| grep -q 'free software' || echo "$URL" >> results.txt
|
||||
|
||||
|
||||
|
||||
#+END_SRC
|
||||
#+BEAMER: \end{onlyenv}
|
||||
|
||||
**** Notes :B_noteNH:
|
||||
:PROPERTIES:
|
||||
:BEAMER_env: noteNH
|
||||
:END:
|
||||
|
||||
We can simplify it by introducing a /variable/.
|
||||
First we assign the URL to a variable named URL.
|
||||
There must be no spaces on either size of the assignment operator,
|
||||
which is the equal sign.
|
||||
We then reference its value by prefixing it with a dollar sign everywhere
|
||||
the URL previously appeared.
|
||||
You should always put variable references in double quotes for safety---that
|
||||
ensures that,
|
||||
if our variable contains a space or other special character,
|
||||
it isn't interpreted by the shell differently than we intended.
|
||||
|
||||
We can also make this command line a bit more concise by using the short
|
||||
name for the =--quiet= flag,
|
||||
which is =-q=.
|
||||
Notice how in =wget= I combined them into =-qO= instead of using two
|
||||
separate dashes with spaces between them.
|
||||
This is optional;
|
||||
if you feel it's easier to read the other way,
|
||||
that's fine.
|
||||
|
||||
Something else feels dirty.
|
||||
We're creating this =speakers.html= file just to pass to =grep=.
|
||||
It's not needed after the fact.
|
||||
In fact,
|
||||
it's just polluting our filesystem.
|
||||
What if we didn't have to create it at all to begin with?
|
||||
|
||||
I'm first going to introduce the notation,
|
||||
and then I'll go into more into why it works.
|
||||
|
||||
If we replace the output file =speakers.html= with a single dash,
|
||||
that tells =wget= to write to standard out.
|
||||
This is normally the default behavior of command line programs,
|
||||
like =grep= and =echo=,
|
||||
but =wget= is a bit different.
|
||||
|
||||
We then omit the =speakers.html= from =grep= entirely.
|
||||
=grep= will read from standard in by default.
|
||||
|
||||
We then connect standard out of =wget= to the standard in of =grep= using a
|
||||
single pipe;
|
||||
this is called a /pipeline/.
|
||||
|
||||
Now that we've freed up some space,
|
||||
let's reformat this slightly to be a bit more readable.
|
||||
|
||||
And now here's the original command we started with,
|
||||
and where we're at now.
|
||||
|
||||
This little bit of abstraction has made our intent even more clear.
|
||||
It can now clearly be read that we're defining a URL,
|
||||
retrieving that URL,
|
||||
searching for a term,
|
||||
and then appending it to a file on a non-match.
|
||||
|
||||
But before we keep going,
|
||||
I want to go back to a point I mentioned previously.
|
||||
|
||||
*** RAW Again: Text is a Universal Interface :B_againframe:
|
||||
:PROPERTIES:
|
||||
:BEAMER_env: againframe
|
||||
:BEAMER_ref: *Text is a Universal Interface
|
||||
:BEAMER_act:
|
||||
:END:
|
||||
|
||||
**** Notes :B_noteNH:
|
||||
:PROPERTIES:
|
||||
:BEAMER_env: noteNH
|
||||
:END:
|
||||
|
||||
Text is a universal interface.
|
||||
|
||||
Notice how we started out our journey manually inspecting text,
|
||||
and began replacing the human part of the workflow at each step with a
|
||||
command.
|
||||
That's because text is something that both humans and computers can work
|
||||
with easily.
|
||||
|
||||
This is a fundamental design principle in the Unix tools that I have begun
|
||||
to present to you.
|
||||
|
||||
*** RAW Pipelines :B_fullframe:
|
||||
:PROPERTIES:
|
||||
:BEAMER_env: fullframe
|
||||
:END:
|
||||
|
||||
#+BEAMER: \fullslidetext
|
||||
/``Expect the output of every program to become the input to another''/
|
||||
|
||||
#+BEAMER: \fullsubtext\hfill
|
||||
---Doug McIlroy (1978)
|
||||
|
||||
**** Notes :B_noteNH:
|
||||
:PROPERTIES:
|
||||
:BEAMER_env: noteNH
|
||||
:END:
|
||||
|
||||
The invention of the Unix pipe is credited to Doug McIlroy.
|
||||
As part of the Unix philosophy,
|
||||
he stated:
|
||||
``expect the output of every program to become the input to another''.
|
||||
|
||||
More broadly,
|
||||
the Unix philosophy can be summarized as:
|
||||
|
||||
*** RAW Summary of the Unix Philosophy :B_fullframe:
|
||||
:PROPERTIES:
|
||||
:BEAMER_env: fullframe
|
||||
:END:
|
||||
|
||||
#+BEAMER: \begingroup\fullslidetext
|
||||
The Unix Philosophy
|
||||
#+BEAMER: \endgroup\subskip
|
||||
|
||||
#+BEGIN_QUOTE
|
||||
This is the Unix philosophy: Write programs that do one thing and do it
|
||||
well. Write programs to work together. Write programs to handle text
|
||||
streams, because that is a universal interface.
|
||||
|
||||
#+BEAMER: \smallsubskip\hfill---Doug McIlroy
|
||||
#+END_QUOTE
|
||||
|
||||
**** Notes :B_noteNH:
|
||||
:PROPERTIES:
|
||||
:BEAMER_env: noteNH
|
||||
:END:
|
||||
|
||||
Up to this point,
|
||||
we have shifted how we communicate with the machine by moving away from a
|
||||
visual interface driven primarily by movement,
|
||||
to a textual interface that puts mind and machine on equal footing.
|
||||
But here we're talking about another profound shift in how we think.
|
||||
|
||||
We start to think of how to decompose problems into small operations that
|
||||
exist as part of a larger pipeline.
|
||||
How to chain them together,
|
||||
transforming the data at each step to make it more suitable for the next.
|
||||
|
||||
|
||||
** Thank You :B_fullframe:
|
||||
:PROPERTIES:
|
||||
:BEAMER_env: fullframe
|
||||
|
|
|
|||
Loading…
Reference in New Issue