2019-03-04 23:39:04 -05:00
|
|
|
|
#+startup: beamer
|
|
|
|
|
#+TITLE: Computational Symbiosis: Methods That Meld Mind and Machine
|
|
|
|
|
#+AUTHOR: Mike Gerwitz
|
|
|
|
|
#+EMAIL: mtg@gnu.org
|
|
|
|
|
#+DATE: LibrePlanet 2019
|
|
|
|
|
#+BEAMER_HEADER: \title{Computational Symbiosis}
|
|
|
|
|
#+BEAMER_HEADER: \subtitle{Methods That Meld Mind and Machine}
|
|
|
|
|
#+OPTIONS: H:3 num:nil toc:nil p:nil todo:nil stat:nil
|
|
|
|
|
#+LaTeX_CLASS: beamer
|
|
|
|
|
#+LaTeX_CLASS_OPTIONS: [presentation]
|
|
|
|
|
#+BEAMER_THEME: Madrid
|
|
|
|
|
#+BEAMER_COLOR_THEME: beetle
|
|
|
|
|
#+BEAMER_HEADER: \input{slides-preamble.tex}
|
2019-03-15 23:51:16 -04:00
|
|
|
|
#+TODO: DEVOID(v) LACKING(l) RAW(r) DRAFT(d) AUGMENT(A) REVIEWED(R) | READY(,) REHEARSED(.)
|
2019-03-04 23:39:04 -05:00
|
|
|
|
#+COLUMNS: %40ITEM %10DURATION{:} %8TODO %BEAMER_ENV(ENVIRONMENT)
|
|
|
|
|
|
|
|
|
|
* Project Notes :noexport:
|
|
|
|
|
** Core Guidelines
|
|
|
|
|
- Blur distinction between user and programmer.
|
|
|
|
|
- Use only examples that non-programmers could come up with and understand.
|
|
|
|
|
- GUIs /are/ sometimes the most efficient means of melding mind and machine.
|
|
|
|
|
- De-emphasize doing everything on a terminal just because its "cool".
|
|
|
|
|
|
2019-03-16 23:45:57 -04:00
|
|
|
|
** Concepts [15/27]
|
2019-03-04 23:39:04 -05:00
|
|
|
|
- [ ] =/sys= and such
|
|
|
|
|
- [ ] Automating multi-step workflows with GNU Make
|
2019-03-16 23:45:57 -04:00
|
|
|
|
- [X] Concurrency ([[*Program Composition]])
|
|
|
|
|
- [X] Connecting graphical and command line (X11 integration) [4/4]
|
|
|
|
|
- [X] Clipboard (image) ([[*More Examples]])
|
2019-03-14 22:57:37 -04:00
|
|
|
|
- [X] Clipboard (text) ([[*Program Composition]])
|
2019-03-16 23:45:57 -04:00
|
|
|
|
- [X] X11 commands
|
|
|
|
|
- [X] Interesting example idea: =import=, =tesseract ss.png - | xsel -ib=
|
2019-03-14 22:57:37 -04:00
|
|
|
|
so that you can select an area of text on the screen and perform
|
2019-03-16 23:45:57 -04:00
|
|
|
|
OCR, then have the result on your clipboard ([[*More Examples]])
|
|
|
|
|
- I've never done this, but it goes to show how trivial it is to
|
|
|
|
|
create something that is seemingly novel.
|
2019-03-04 23:39:04 -05:00
|
|
|
|
- [ ] Discoverability
|
|
|
|
|
- Example: I used GIMP for editing some images for this talk because I
|
|
|
|
|
thought it'd be faster to discover how to that way vs. ImageMagick docs.
|
|
|
|
|
- [ ] Editor should keep up with your thoughts
|
|
|
|
|
- Emphasize muscle memory
|
|
|
|
|
- [-] Efficiency gains of light use of keybindings [2/3]
|
|
|
|
|
- [X] Seldom change ([[*Browser Topics][Browser Topics]])
|
|
|
|
|
- [X] Some are universal (between programs) ([[*Browser Topics][Browser Topics]])
|
|
|
|
|
- [ ] Some context-sensitive and Just Work without having to hunt through
|
|
|
|
|
context-specific menus
|
|
|
|
|
- [ ] Emphasize free software community and how OS is built by so many
|
|
|
|
|
contributors
|
|
|
|
|
- Symbiosis is not only with machine, but community
|
2019-03-16 23:45:57 -04:00
|
|
|
|
- [X] Globbing
|
|
|
|
|
- [X] GnuPG and smart cards ([[*More Examples]])
|
|
|
|
|
- [X] Incremental, iterative, interactive development ([[*Program Composition]])
|
2019-03-10 22:38:26 -04:00
|
|
|
|
- [X] Looping ([[*Perspective Topics][Perspective Topics]])
|
2019-03-04 23:39:04 -05:00
|
|
|
|
- [ ] McIlroy and Knuth
|
|
|
|
|
- [X] Mouse has dependency on positioning of UI elements ([[*Browser Topics][Browser Topics]])
|
|
|
|
|
- Changes over time, subject to user/system preferences, etc
|
|
|
|
|
- [ ] Multiplexing with GNU Screen
|
2019-03-14 22:57:37 -04:00
|
|
|
|
- [X] Output redirection ([[*A New Perspective]])
|
2019-03-04 23:39:04 -05:00
|
|
|
|
- [ ] Phone as an example of a device that best works with touch and GUIs
|
|
|
|
|
and sacrifices practical freedom.
|
|
|
|
|
- Requires specialized knowledge and is inconvenient to work with.
|
2019-03-10 22:38:26 -04:00
|
|
|
|
- [X] Pipelines instead of intermediate files ([[*Perspective Topics][Perspective Topics]]).
|
2019-03-16 23:45:57 -04:00
|
|
|
|
- [X] Practically exercising software freedom [2/2]
|
|
|
|
|
- [X] Users need to be able to convey their thoughts to the computer
|
2019-03-04 23:39:04 -05:00
|
|
|
|
without being programmers
|
2019-03-16 23:45:57 -04:00
|
|
|
|
- [X] Concise primitives / building blocks
|
2019-03-15 00:46:44 -04:00
|
|
|
|
- [ ] Readline (history and editing) [0/2]
|
|
|
|
|
- [ ] Develop as you go, perhaps just referencing history early on
|
|
|
|
|
- [ ] Transfer commands from history into scripts and aliases for re-use
|
2019-03-10 22:38:26 -04:00
|
|
|
|
- [X] Regular expressions ([[*Perspective Topics][Perspective Topics]])
|
2019-03-16 23:45:57 -04:00
|
|
|
|
- [X] Remote commands via SSH
|
2019-03-04 23:39:04 -05:00
|
|
|
|
- [X] Text as a universal interface
|
2019-03-10 22:38:26 -04:00
|
|
|
|
- [X] All programs can work with one-another ([[*Perspective Topics][Perspective Topics]])
|
2019-03-04 23:39:04 -05:00
|
|
|
|
- [X] Can replace any part of the process with a human---no difference
|
|
|
|
|
between input from a program vs. from a keyboard. ([[*Browser Topics][Browser Topics]])
|
2019-03-10 22:38:26 -04:00
|
|
|
|
- [X] The Unix Philosophy ([[*Perspective Topics][Perspective Topics]])
|
2019-03-04 23:39:04 -05:00
|
|
|
|
- [X] Using keybindings in a GUI with no mouse ([[*Browser Topics][Browser Topics]])
|
|
|
|
|
- Including transferring data between programs
|
|
|
|
|
- [ ] Using macros to script keypresses (Vim)
|
|
|
|
|
- [ ] Version control
|
2019-03-08 22:38:21 -05:00
|
|
|
|
|
2019-03-04 23:39:04 -05:00
|
|
|
|
** Programs [0/4]
|
2019-03-16 23:45:57 -04:00
|
|
|
|
- [-] Coreutils / Common [12/15]
|
2019-03-14 22:57:37 -04:00
|
|
|
|
- [X] =cut=
|
2019-03-04 23:39:04 -05:00
|
|
|
|
- [ ] =column=
|
|
|
|
|
- [ ] =diff=
|
2019-03-14 22:57:37 -04:00
|
|
|
|
- [X] =echo=
|
2019-03-16 23:45:57 -04:00
|
|
|
|
- [X] =find=
|
|
|
|
|
- [X] =gpg=
|
2019-03-14 22:57:37 -04:00
|
|
|
|
- [X] =grep=
|
2019-03-04 23:39:04 -05:00
|
|
|
|
- [ ] =make= / Makefiles
|
2019-03-16 23:45:57 -04:00
|
|
|
|
- [X] =man= and =info=
|
2019-03-14 22:57:37 -04:00
|
|
|
|
- [X] =sort=
|
2019-03-16 23:45:57 -04:00
|
|
|
|
- [X] =ssh=
|
2019-03-14 22:57:37 -04:00
|
|
|
|
- [X] =uniq=
|
2019-03-16 23:45:57 -04:00
|
|
|
|
- [X] =watch=
|
2019-03-14 22:57:37 -04:00
|
|
|
|
- [X] =wc=
|
|
|
|
|
- [X] =wget= / =curl=
|
2019-03-16 23:45:57 -04:00
|
|
|
|
- [-] Miscellaneous [5/8]
|
2019-03-04 23:39:04 -05:00
|
|
|
|
- [ ] =awk=
|
2019-03-16 23:45:57 -04:00
|
|
|
|
- [X] =cowsay= / =figlet=
|
2019-03-08 21:32:38 -05:00
|
|
|
|
- [ ] =dict=
|
2019-03-14 22:57:37 -04:00
|
|
|
|
- [X] =espeak=
|
2019-03-04 23:39:04 -05:00
|
|
|
|
- [ ] =git=
|
2019-03-14 22:57:37 -04:00
|
|
|
|
- [X] =mail=
|
|
|
|
|
- [X] =torify=
|
|
|
|
|
- [X] =sed=
|
2019-03-16 23:45:57 -04:00
|
|
|
|
- [-] X11 / Graphical [3/5]
|
|
|
|
|
- [X] =convert=
|
2019-03-04 23:39:04 -05:00
|
|
|
|
- [ ] =i3= / =xmonad= / etc
|
2019-03-16 23:45:57 -04:00
|
|
|
|
- [X] =import=
|
2019-03-04 23:39:04 -05:00
|
|
|
|
- [ ] =setxkbmap=
|
2019-03-14 22:57:37 -04:00
|
|
|
|
- [X] =xclip= / =xsel=
|
2019-03-04 23:39:04 -05:00
|
|
|
|
- [ ] Editors / Multiplexing [0/4]
|
|
|
|
|
- [ ] =emacs=
|
|
|
|
|
- [ ] =nano=
|
|
|
|
|
- [ ] =vim=
|
|
|
|
|
- [ ] =screen=
|
|
|
|
|
|
2019-03-25 12:53:44 -04:00
|
|
|
|
* REHEARSED Slides [5/5]
|
2019-03-04 23:39:04 -05:00
|
|
|
|
:PROPERTIES:
|
2019-03-20 00:01:10 -04:00
|
|
|
|
:ID: slides
|
2019-03-04 23:39:04 -05:00
|
|
|
|
:END:
|
|
|
|
|
** Summary :noexport:
|
|
|
|
|
#+BEGIN: columnview :hlines 2 :maxlevel 3 :indent t :id slides
|
2019-03-25 12:53:44 -04:00
|
|
|
|
| ITEM | DURATION | TODO | ENVIRONMENT |
|
|
|
|
|
|-------------------------------------------------------+----------+-----------+-------------|
|
|
|
|
|
| Slides | 0:39:53 | REHEARSED | |
|
|
|
|
|
|-------------------------------------------------------+----------+-----------+-------------|
|
|
|
|
|
| \_ Summary | | | |
|
|
|
|
|
|-------------------------------------------------------+----------+-----------+-------------|
|
|
|
|
|
| \_ Introduction | 0:01:24 | REHEARSED | |
|
|
|
|
|
| \_ Spoken Intro | 0:00:25 | REHEARSED | note |
|
|
|
|
|
| \_ Choreographed Workflows | 00:00:20 | REHEARSED | fullframe |
|
|
|
|
|
| \_ Practical Freedom | 0:00:40 | REHEARSED | fullframe |
|
|
|
|
|
|-------------------------------------------------------+----------+-----------+-------------|
|
|
|
|
|
| \_ Practical Example: Web Browser | 0:06:34 | REHEARSED | |
|
|
|
|
|
| \_ Browser Topics | | | |
|
|
|
|
|
| \_ Example: Web Browser | 0:00:10 | REHEARSED | frame |
|
|
|
|
|
| \_ Finding Text (Mouse-Driven GUI Interaction) | 0:01:00 | REHEARSED | frame |
|
|
|
|
|
| \_ GUIs Change Over Time | 0:00:45 | REHEARSED | frame |
|
|
|
|
|
| \_ Muscle Memory | 0:00:40 | REHEARSED | fullframe |
|
|
|
|
|
| \_ A Research Task | 0:00:20 | REHEARSED | fullframe |
|
|
|
|
|
| \_ Executing the Research Task | 0:02:05 | REHEARSED | frame |
|
|
|
|
|
| \_ GUIs of a Feather | 0:00:35 | REHEARSED | fullframe |
|
|
|
|
|
| \_ Macro-Like Keyboard Instructions | 0:01:00 | REHEARSED | fullframe |
|
|
|
|
|
|-------------------------------------------------------+----------+-----------+-------------|
|
|
|
|
|
| \_ A New Perspective | 0:12:48 | REHEARSED | |
|
|
|
|
|
| \_ Perspective Topics | | | |
|
|
|
|
|
| \_ Secrets? | 0:00:55 | REHEARSED | fullframe |
|
|
|
|
|
| \_ Lifting the Curtain | 0:00:35 | REHEARSED | frame |
|
|
|
|
|
| \_ Text | 0:00:35 | REHEARSED | fullframe |
|
|
|
|
|
| \_ Text is a Universal Interface | 0:01:10 | REHEARSED | fullframe |
|
|
|
|
|
| \_ The Shell Command Prompt | 0:00:55 | REHEARSED | frame |
|
|
|
|
|
| \_ Eliminating the Web Browser | 0:01:04 | REHEARSED | frame |
|
|
|
|
|
| \_ Browser vs. =wget= Comparison | 0:00:20 | REHEARSED | frame |
|
|
|
|
|
| \_ Finding Text on the Command Line | 0:01:00 | REHEARSED | frame |
|
|
|
|
|
| \_ A More Gentle Reply | 0:01:00 | REHEARSED | frame |
|
|
|
|
|
| \_ Writing to Files (Redirection) | 0:00:55 | REHEARSED | frame |
|
|
|
|
|
| \_ Starting Our List | 0:01:00 | REHEARSED | fullframe |
|
|
|
|
|
| \_ Command Refactoring | 0:02:05 | REHEARSED | fullframe |
|
|
|
|
|
| \_ Again: Text is a Universal Interface | 0:00:20 | REHEARSED | againframe |
|
|
|
|
|
| \_ Pipelines | 0:00:15 | REHEARSED | fullframe |
|
|
|
|
|
| \_ Summary of the Unix Philosophy | 0:00:40 | REHEARSED | fullframe |
|
|
|
|
|
|-------------------------------------------------------+----------+-----------+-------------|
|
|
|
|
|
| \_ Program Composition | 0:09:15 | REHEARSED | |
|
|
|
|
|
| \_ Composition Topics | | | |
|
|
|
|
|
| \_ LP Sessions | 0:02:30 | REHEARSED | fullframe |
|
|
|
|
|
| \_ Interactive, Incremental, Iterative Development | 0:01:10 | REHEARSED | fullframe |
|
|
|
|
|
| \_ Discovering URLs | 0:02:15 | REHEARSED | fullframe |
|
|
|
|
|
| \_ Go Grab a Coffee | 0:00:15 | REHEARSED | fullframe |
|
|
|
|
|
| \_ Async Processes | 0:00:45 | REHEARSED | fullframe |
|
|
|
|
|
| \_ Executable Shell Script and Concurrency | 0:01:30 | REHEARSED | fullframe |
|
|
|
|
|
| \_ Execution Time | 0:00:50 | REHEARSED | frame |
|
|
|
|
|
|-------------------------------------------------------+----------+-----------+-------------|
|
|
|
|
|
| \_ More Examples | 0:09:52 | REHEARSED | |
|
|
|
|
|
| \_ More Example Topics | | | |
|
|
|
|
|
| \_ Resize Images | 0:00:30 | REHEARSED | frame |
|
|
|
|
|
| \_ Password Generation | 0:00:45 | REHEARSED | frame |
|
|
|
|
|
| \_ Password Manager | 0:01:39 | REHEARSED | frame |
|
|
|
|
|
| \_ Remote Password Manager With 2FA | 0:01:30 | REHEARSED | frame |
|
|
|
|
|
| \_ Taking Screenshots | 0:00:35 | REHEARSED | frame |
|
|
|
|
|
| \_ Screenshot OCR | 0:01:00 | REHEARSED | frame |
|
|
|
|
|
| \_ Full Circle | 0:02:24 | REHEARSED | fullframe |
|
|
|
|
|
| \_ Getting Help | 0:01:30 | REHEARSED | frame |
|
|
|
|
|
|-------------------------------------------------------+----------+-----------+-------------|
|
|
|
|
|
| \_ Thank You | 00:00:01 | | fullframe |
|
2019-03-04 23:39:04 -05:00
|
|
|
|
#+END:
|
|
|
|
|
|
2019-03-25 12:53:44 -04:00
|
|
|
|
** REHEARSED Introduction [3/3]
|
|
|
|
|
*** REHEARSED Spoken Intro :B_note:
|
2019-03-04 23:39:04 -05:00
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: note
|
2019-03-25 12:53:44 -04:00
|
|
|
|
:DURATION: 0:00:25
|
2019-03-04 23:39:04 -05:00
|
|
|
|
:END:
|
2019-03-20 00:01:10 -04:00
|
|
|
|
|
2019-03-04 23:39:04 -05:00
|
|
|
|
Hello, everyone!
|
|
|
|
|
|
|
|
|
|
My name is Mike Gerwitz.
|
|
|
|
|
I am a free software hacker and activist with a focus on user privacy and
|
|
|
|
|
security.
|
2019-03-25 12:53:44 -04:00
|
|
|
|
I'm also a GNU Maintainer and volunteer.
|
2019-03-04 23:39:04 -05:00
|
|
|
|
I have about twenty years of programming experience,
|
2019-03-17 13:42:42 -04:00
|
|
|
|
half of that professionally.
|
2019-03-08 22:38:21 -05:00
|
|
|
|
|
2019-03-17 13:42:42 -04:00
|
|
|
|
And I've been a computer user for longer.
|
2019-03-04 23:39:04 -05:00
|
|
|
|
So I've been around long enough to see a decent evolution in how we interact
|
2019-03-17 13:42:42 -04:00
|
|
|
|
with machines.
|
2019-03-04 23:39:04 -05:00
|
|
|
|
|
|
|
|
|
|
2019-03-25 12:53:44 -04:00
|
|
|
|
*** REHEARSED Choreographed Workflows :B_fullframe:
|
2019-03-04 23:39:04 -05:00
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: fullframe
|
2019-03-20 22:47:20 -04:00
|
|
|
|
:DURATION: 00:00:20
|
2019-03-04 23:39:04 -05:00
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
#+BEAMER: \fullslidetext
|
|
|
|
|
Choreographed Workflows
|
|
|
|
|
|
2019-03-17 13:42:42 -04:00
|
|
|
|
**** Notes :B_noteNH:
|
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: noteNH
|
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
What we have tended toward over the years are interfaces that try to cater
|
|
|
|
|
to as many people as possible by providing carefully choreographed
|
|
|
|
|
workflows that think /for/ you.
|
|
|
|
|
And I don't deny that this has been a useful method for making computers
|
|
|
|
|
accessible to huge numbers of people.
|
|
|
|
|
But it's important to understand where this trend falls short.
|
|
|
|
|
|
|
|
|
|
|
2019-03-25 12:53:44 -04:00
|
|
|
|
*** REHEARSED Practical Freedom :B_fullframe:
|
2019-03-04 23:39:04 -05:00
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: fullframe
|
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
#+BEAMER: \fullslidetext
|
|
|
|
|
Practical Freedom
|
|
|
|
|
|
2019-03-17 13:42:42 -04:00
|
|
|
|
**** Notes :B_noteNH:
|
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: noteNH
|
2019-03-21 22:52:39 -04:00
|
|
|
|
:DURATION: 00:00:40
|
2019-03-17 13:42:42 -04:00
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
This is a talk about /practical/ freedoms---an
|
2019-03-20 00:01:10 -04:00
|
|
|
|
issue separate from but /requiring/ software freedom.
|
2019-03-17 13:42:42 -04:00
|
|
|
|
If developers are thinking /for/ us and guiding us in our computing,
|
|
|
|
|
then we're limited to preconceived workflows.
|
|
|
|
|
This leaves immense power in the hands of developers even if software is
|
|
|
|
|
free,
|
2019-03-21 22:52:39 -04:00
|
|
|
|
because average users are stuck asking them to implement changes,
|
|
|
|
|
or footing the bill for someone else to do so.
|
2019-03-17 13:42:42 -04:00
|
|
|
|
|
|
|
|
|
My goal here is to blur those lines between ``user'' and ``programmer'' and
|
|
|
|
|
show you how users can be empowered to take control of their computing in
|
2019-03-20 00:01:10 -04:00
|
|
|
|
practical and powerful ways.
|
2019-03-20 22:47:20 -04:00
|
|
|
|
But it does require a different way of thinking.
|
2019-03-17 13:42:42 -04:00
|
|
|
|
|
2019-03-20 22:47:20 -04:00
|
|
|
|
To begin,
|
2019-03-25 12:53:44 -04:00
|
|
|
|
let's start by exploring something that I think most people can relate to
|
|
|
|
|
using.
|
2019-03-17 13:42:42 -04:00
|
|
|
|
|
2019-03-04 23:39:04 -05:00
|
|
|
|
|
2019-03-25 12:53:44 -04:00
|
|
|
|
** REHEARSED Practical Example: Web Browser [8/8]
|
2019-03-10 22:38:26 -04:00
|
|
|
|
*** Browser Topics [3/3] :noexport:
|
2019-03-04 23:39:04 -05:00
|
|
|
|
This example is the main segue from GUIs into the utility of keybindings
|
|
|
|
|
into shell, so it has to be compelling. I chose something that is
|
|
|
|
|
frequently done by users: visiting webpages and searching for text.
|
|
|
|
|
|
|
|
|
|
- [X] Most webpages include styling, images, and other media that are best
|
|
|
|
|
represented in a GUI.
|
|
|
|
|
- Of course, I'm a firm believer in progressive enhancement, but this
|
|
|
|
|
isn't a web development talk.
|
2019-03-05 01:06:02 -05:00
|
|
|
|
- [X] Is it good for finding a word or phrase on a page? [4/4]
|
2019-03-04 23:39:04 -05:00
|
|
|
|
- [X] Menu -> Find in This Page -> Type. Sure.
|
2019-03-05 01:06:02 -05:00
|
|
|
|
- [X] Instructing a user how to do this is inherently visual. Comment on
|
2019-03-04 23:39:04 -05:00
|
|
|
|
my need to highlight areas of screenshots.
|
2019-03-05 01:06:02 -05:00
|
|
|
|
- [X] More experienced users may prefer =C-f=.
|
2019-03-04 23:39:04 -05:00
|
|
|
|
- Emphasize the motion with my hand. Explain how that mental association
|
|
|
|
|
is made. Really focus on this.
|
2019-03-05 01:06:02 -05:00
|
|
|
|
- [X] Location of menu has changed over time (show screenshots), but =C-f=
|
2019-03-04 23:39:04 -05:00
|
|
|
|
has been unchanged for decades.
|
2019-03-08 22:38:21 -05:00
|
|
|
|
- [X] What if I had a list of five webpages and I wanted to get rid of the
|
|
|
|
|
ones that didn't have that word? [4/4]
|
|
|
|
|
- [X] Mouse and GUI way:
|
2019-03-04 23:39:04 -05:00
|
|
|
|
- Open each page in a new tab.
|
|
|
|
|
- Menu -> Find on page -> Type.
|
|
|
|
|
- Found, copy URL, open a word processor, paste.
|
|
|
|
|
- Go back, click X on the tab.
|
|
|
|
|
- Repeat.
|
2019-03-08 22:38:21 -05:00
|
|
|
|
- [X] Then show keybinding way:
|
2019-03-04 23:39:04 -05:00
|
|
|
|
- *Demonstrate this on the LP laptop.*
|
|
|
|
|
- Open programs with =M-F2=
|
|
|
|
|
- Open each page in a new tab (=C-t=)
|
|
|
|
|
- =C-f ESC=
|
|
|
|
|
- =C-l C-c=
|
|
|
|
|
- =M-TAB C-v RET=
|
|
|
|
|
- =M-TAB C-w=
|
|
|
|
|
- Repeat
|
2019-03-08 22:38:21 -05:00
|
|
|
|
- [X] This would work with nearly /any/ combination of web browser and
|
2019-03-04 23:39:04 -05:00
|
|
|
|
text editor.
|
2019-03-08 22:38:21 -05:00
|
|
|
|
- [X] But what if I had 10? 100? 1000? This is still tedious and slow.
|
2019-03-04 23:39:04 -05:00
|
|
|
|
|
2019-03-25 12:53:44 -04:00
|
|
|
|
*** REHEARSED Example: Web Browser :B_frame:
|
2019-03-04 23:39:04 -05:00
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: frame
|
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
[[file:images/web-browser.png]]
|
|
|
|
|
|
|
|
|
|
**** Notes :B_noteNH:
|
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: noteNH
|
2019-03-25 12:53:44 -04:00
|
|
|
|
:DURATION: 00:00:10
|
2019-03-04 23:39:04 -05:00
|
|
|
|
:END:
|
|
|
|
|
|
2019-03-20 00:01:10 -04:00
|
|
|
|
The only GUI I use on a day-to-day basis is my web browser.
|
2019-03-04 23:39:04 -05:00
|
|
|
|
In this case,
|
|
|
|
|
GNU Icecat,
|
|
|
|
|
which is a Firefox derivative.
|
|
|
|
|
This is a screenshot of an admittedly staged session,
|
|
|
|
|
and contains a number of addons.
|
2019-03-12 00:35:14 -04:00
|
|
|
|
|
2019-03-25 12:53:44 -04:00
|
|
|
|
*** REHEARSED Finding Text (Mouse-Driven GUI Interaction) :B_frame:
|
2019-03-04 23:39:04 -05:00
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: frame
|
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
**** Images :B_columns:
|
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: columns
|
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
***** Left :B_column:
|
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: column
|
|
|
|
|
:BEAMER_col: 0.30
|
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
#+ATTR_LATEX: :width 1.5in
|
|
|
|
|
[[file:images/ff-find-menu.png]]
|
|
|
|
|
|
|
|
|
|
***** Right :B_column:
|
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: column
|
|
|
|
|
:BEAMER_col: 0.70
|
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
#+ATTR_LATEX: :width 3in
|
|
|
|
|
[[file:images/ff-find-matches.png]]
|
|
|
|
|
|
|
|
|
|
**** Notes :B_noteNH:
|
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: noteNH
|
2019-03-20 23:49:14 -04:00
|
|
|
|
:DURATION: 00:01:00
|
2019-03-04 23:39:04 -05:00
|
|
|
|
:END:
|
|
|
|
|
|
2019-03-20 23:49:14 -04:00
|
|
|
|
A common operation is to search for a word or phrase,
|
|
|
|
|
like shown here.
|
2019-03-04 23:39:04 -05:00
|
|
|
|
|
|
|
|
|
Now,
|
2019-03-20 00:01:10 -04:00
|
|
|
|
how exactly to do this with a mouse varies depending on what browser
|
2019-03-04 23:39:04 -05:00
|
|
|
|
you're using,
|
2019-03-20 00:01:10 -04:00
|
|
|
|
but here I highlighted the steps in a modern IceCat or Firefox.
|
2019-03-04 23:39:04 -05:00
|
|
|
|
You start by clicking on the little hamburger, hotdog, or
|
|
|
|
|
whatever-you-want-to-call-it menu in the upper-right,
|
|
|
|
|
and then click on "Find in This Page" within the popup.
|
|
|
|
|
This then opens a bar at the bottom of the page with an area to type the
|
|
|
|
|
word or phrase you're searching for.
|
|
|
|
|
It highlights and scrolls to the first match as you type,
|
|
|
|
|
and has a button to highlight all results.
|
|
|
|
|
It also shows the number of results off to the right.
|
|
|
|
|
It's a simple,
|
|
|
|
|
yet powerful mechanism that is pretty easy to use.
|
2019-03-08 22:38:21 -05:00
|
|
|
|
|
2019-03-20 00:01:10 -04:00
|
|
|
|
Is this an efficient means to communicate your intent?
|
2019-03-04 23:39:04 -05:00
|
|
|
|
If you're just searching for a name or concept,
|
|
|
|
|
sure,
|
2019-03-20 00:01:10 -04:00
|
|
|
|
it's not so bad.
|
2019-03-04 23:39:04 -05:00
|
|
|
|
|
|
|
|
|
But notice how I had to convey these steps to you.
|
|
|
|
|
I had to take screenshots and highlight where to click with the mouse.
|
|
|
|
|
Since a GUI is inherently very visual,
|
|
|
|
|
so are the instructions on how to use it.
|
|
|
|
|
There is no canonical representation for these instructions,
|
|
|
|
|
because it involves clicking on elements that have no clear name to the
|
|
|
|
|
user.
|
|
|
|
|
|
2019-03-25 12:53:44 -04:00
|
|
|
|
*** REHEARSED GUIs Change Over Time :B_frame:
|
2019-03-04 23:39:04 -05:00
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: frame
|
|
|
|
|
:END:
|
|
|
|
|
**** Images :B_columns:
|
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: columns
|
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
***** Left :B_column:
|
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: column
|
|
|
|
|
:BEAMER_col: 0.50
|
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
#+ATTR_LATEX: :width 1.5in
|
|
|
|
|
[[file:images/ff-find-menu.png]]
|
|
|
|
|
|
|
|
|
|
***** Right :B_column:
|
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: column
|
|
|
|
|
:BEAMER_col: 0.50
|
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
#+ATTR_LATEX: :width 1.5in
|
|
|
|
|
[[file:images/ff-edit-find.png]]
|
|
|
|
|
|
|
|
|
|
#+BEAMER: \uncover<2>{\Large
|
|
|
|
|
Ctrl+F
|
|
|
|
|
#+BEAMER: }
|
|
|
|
|
|
|
|
|
|
**** Notes :B_noteNH:
|
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: noteNH
|
2019-03-12 00:35:14 -04:00
|
|
|
|
:DURATION: 00:00:45
|
2019-03-04 23:39:04 -05:00
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
Another difficult thing is: GUIs change over time.
|
2019-03-12 00:35:14 -04:00
|
|
|
|
I'm sure many people here remember earlier versions of Firefox that didn't
|
|
|
|
|
have the hamburger menu,
|
2019-03-04 23:39:04 -05:00
|
|
|
|
where the Find menu option was in the Edit menu.
|
2019-03-20 23:49:14 -04:00
|
|
|
|
Those old menus do still exist if you hit Alt.
|
|
|
|
|
I miss the old menus.
|
2019-03-04 23:39:04 -05:00
|
|
|
|
Saying "Go to Edit - Find" is pretty clear,
|
|
|
|
|
and those menu positions were always in the same place across the entire
|
|
|
|
|
desktop environment.
|
|
|
|
|
Now individual programs may vary in the their user experience.
|
|
|
|
|
|
|
|
|
|
But do you notice something in common between these two screenshots?
|
|
|
|
|
There's something that /hasn't/ changed over time---something
|
|
|
|
|
that has been the same for /decades/!
|
|
|
|
|
=Ctrl+F=.
|
|
|
|
|
|
|
|
|
|
When you type =Ctrl+F=,
|
2019-03-05 01:06:02 -05:00
|
|
|
|
it immediately opens that search bar and gives focus to the textbox,
|
|
|
|
|
so you can just start typing.
|
|
|
|
|
|
2019-03-25 12:53:44 -04:00
|
|
|
|
*** REHEARSED Muscle Memory :B_fullframe:
|
2019-03-05 01:06:02 -05:00
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: fullframe
|
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
#+BEAMER: \fullslidetext
|
|
|
|
|
Muscle Memory
|
|
|
|
|
|
|
|
|
|
#+BEAMER: \fullsubtext
|
|
|
|
|
Visual \Rightarrow Tactile
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
**** Notes :B_noteNH:
|
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: noteNH
|
2019-03-12 00:35:14 -04:00
|
|
|
|
:DURATION: 00:00:40
|
2019-03-05 01:06:02 -05:00
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
With a mouse and a GUI,
|
|
|
|
|
interaction is driven by visual indicators.
|
2019-03-12 00:35:14 -04:00
|
|
|
|
The position of your hand on the mousepad or your fingers on a touchpad
|
|
|
|
|
isn't meaningful,
|
|
|
|
|
because your mouse cursor could be anywhere on the screen at any given
|
|
|
|
|
time;
|
|
|
|
|
your /eyes/ provide the context.
|
2019-03-05 01:06:02 -05:00
|
|
|
|
|
|
|
|
|
But by hitting =Ctrl+F=,
|
2019-03-20 23:49:14 -04:00
|
|
|
|
we have switched our mode of interaction.
|
2019-03-05 01:06:02 -05:00
|
|
|
|
It's now /tactile/.
|
|
|
|
|
You associate a finger placement;
|
|
|
|
|
a motion;
|
2019-03-20 00:01:10 -04:00
|
|
|
|
and the feeling of the keys being pressed beneath your fingers
|
2019-03-05 01:06:02 -05:00
|
|
|
|
with an action---finding
|
|
|
|
|
something.
|
|
|
|
|
You develop muscle memory.
|
|
|
|
|
|
2019-03-12 00:35:14 -04:00
|
|
|
|
/<Repeatedly make motion with hand and fingers like a madman during the
|
|
|
|
|
above paragraph.>/
|
2019-03-05 01:06:02 -05:00
|
|
|
|
|
2019-03-25 12:53:44 -04:00
|
|
|
|
*** REHEARSED A Research Task :B_fullframe:
|
2019-03-08 22:38:21 -05:00
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: fullframe
|
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
#+BEAMER: \fullslidetext
|
|
|
|
|
Research Task:
|
|
|
|
|
|
|
|
|
|
#+BEAMER: \fullsubtext
|
|
|
|
|
Given a list of webpage URLs
|
|
|
|
|
#+BEAMER: \smallsubskip
|
|
|
|
|
find all that /do not/ contain ``free software''
|
|
|
|
|
|
|
|
|
|
**** Notes :B_noteNH:
|
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: noteNH
|
2019-03-20 23:49:14 -04:00
|
|
|
|
:DURATION: 00:00:20
|
2019-03-08 22:38:21 -05:00
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
Let's explore a fairly simple research task together.
|
2019-03-20 00:01:10 -04:00
|
|
|
|
Let's say I email you a handfull of URLs---maybe
|
|
|
|
|
5 or 10 of them---that
|
2019-03-08 22:38:21 -05:00
|
|
|
|
are articles about software or technology.
|
|
|
|
|
And I want you to come up with a list of the webpages that /do not/ contain
|
|
|
|
|
the phrase ``free software'' so that I can get a better idea of ones to
|
|
|
|
|
focus my activism on.
|
|
|
|
|
|
|
|
|
|
How might we approach this problem as an average user?
|
|
|
|
|
|
2019-03-25 12:53:44 -04:00
|
|
|
|
*** REHEARSED Executing the Research Task :B_frame:
|
2019-03-08 22:38:21 -05:00
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: frame
|
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
**** Approaches :B_columns:
|
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: columns
|
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
***** Mouse :B_column:
|
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: column
|
|
|
|
|
:BEAMER_col: 0.50
|
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
_Mouse_
|
|
|
|
|
|
|
|
|
|
1. Click `+' for each new tab, enter\nbsp{}URL
|
|
|
|
|
2. Menu \rightarrow Find in This Page
|
|
|
|
|
3. Type ``free software''
|
|
|
|
|
4. If found, go to #9
|
|
|
|
|
5. If /not/ found, highlight URL, right-click, copy
|
|
|
|
|
6. Click on text editor
|
|
|
|
|
7. Right-click, paste URL, hit =RET= for newline
|
|
|
|
|
8. Click on web browser
|
|
|
|
|
9. Click `X' on tab, go to #2
|
|
|
|
|
|
|
|
|
|
******* Notes :B_noteNH:
|
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: noteNH
|
2019-03-20 23:49:14 -04:00
|
|
|
|
:DURATION: 00:01:15
|
2019-03-08 22:38:21 -05:00
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
/(Perhaps I should demonstrate this right away rather than reading through
|
|
|
|
|
the list first, to save time?)/
|
|
|
|
|
|
|
|
|
|
Let's first use the mouse as many users probably would.
|
|
|
|
|
To set up,
|
|
|
|
|
let's open each URL in a new tab.
|
|
|
|
|
We click on the little `+' icon for a new tab and then enter the URL,
|
|
|
|
|
once for each webpage,
|
|
|
|
|
perhaps copying the URL from the email message.
|
|
|
|
|
Once we're all set up,
|
|
|
|
|
we don't care about the email anymore,
|
|
|
|
|
but we need a place to store our results,
|
|
|
|
|
so we open a text editor to paste URLs into.
|
|
|
|
|
|
|
|
|
|
Now, for each tab,
|
|
|
|
|
we click on the little hamburger menu,
|
|
|
|
|
click on ``Find in This Page'',
|
|
|
|
|
and then type ``free software''.
|
|
|
|
|
If we /do not/ see a result,
|
|
|
|
|
we move our mouse to the location bar,
|
|
|
|
|
click on it to highlight the URL,
|
2019-03-20 23:49:14 -04:00
|
|
|
|
right-click, copy,
|
2019-03-08 22:38:21 -05:00
|
|
|
|
click on the text editor to give it focus,
|
2019-03-20 23:49:14 -04:00
|
|
|
|
right-click on the editor, ``Paste'',
|
2019-03-08 22:38:21 -05:00
|
|
|
|
and then hit the return key to move to the next line.
|
|
|
|
|
We then go back to the web browser.
|
|
|
|
|
If we /do/ see a result,
|
|
|
|
|
we skip copying over the URL.
|
2019-03-12 00:35:14 -04:00
|
|
|
|
In either case,
|
|
|
|
|
we then close the tab by clicking on the `X'.
|
2019-03-08 22:38:21 -05:00
|
|
|
|
|
2019-03-12 00:35:14 -04:00
|
|
|
|
And then we repeat this for each tab,
|
2019-03-08 22:38:21 -05:00
|
|
|
|
until they have all been closed.
|
|
|
|
|
When we're done,
|
|
|
|
|
whatever is in our text editor is the list of URLs of webpages that do not
|
|
|
|
|
reference ``free software'',
|
|
|
|
|
and we're done.
|
|
|
|
|
|
|
|
|
|
Simple enough,
|
|
|
|
|
right?
|
|
|
|
|
But it's a bit of a pain in the ass.
|
|
|
|
|
All this clicking around doesn't really /feel/ like we're melding mind and
|
|
|
|
|
machine, does it?
|
|
|
|
|
|
|
|
|
|
What if we used our =Ctrl+F= trick?
|
2019-03-20 23:49:14 -04:00
|
|
|
|
That saves us a couple clicks,
|
|
|
|
|
but can we do better?
|
2019-03-08 22:38:21 -05:00
|
|
|
|
|
|
|
|
|
***** Keyboard :B_column:
|
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: column
|
|
|
|
|
:BEAMER_col: 0.50
|
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
#+BEAMER: \uncover<2>{
|
|
|
|
|
_Keyboard_
|
|
|
|
|
|
|
|
|
|
1. *=Ctrl+T=* for each new tab, enter\nbsp{}URL
|
|
|
|
|
2. *=Ctrl+F=* to find
|
|
|
|
|
3. Type ``free software''
|
|
|
|
|
4. If found, go to #9
|
|
|
|
|
5. If /not/ found, *=Ctrl+L Ctrl+C=* to copy URL
|
|
|
|
|
6. *=Alt+Tab=* to text editor
|
|
|
|
|
7. *=Ctrl+V RET=* to paste URL and add newline
|
|
|
|
|
8. *=Alt+Tab=* to web browser
|
|
|
|
|
9. *=Ctrl+W=* to close tab, go to #2
|
|
|
|
|
#+BEAMER: }
|
|
|
|
|
|
|
|
|
|
******* Notes :B_noteNH:
|
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: noteNH
|
2019-03-25 12:53:44 -04:00
|
|
|
|
:DURATION: 00:00:50
|
2019-03-08 22:38:21 -05:00
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
Fortunately we have /many/ more keybindings at our disposal!
|
|
|
|
|
|
|
|
|
|
We'll start with opening each new tab with =Ctrl+T= instead of clicking on
|
|
|
|
|
`+' with the mouse.
|
|
|
|
|
/(Maybe show copying the URL from the email without the mouse?)/
|
|
|
|
|
|
|
|
|
|
To open our text editor,
|
2019-03-20 00:01:10 -04:00
|
|
|
|
we'll use =Alt+F4=.
|
2019-03-08 22:38:21 -05:00
|
|
|
|
|
|
|
|
|
Once we're all set up,
|
|
|
|
|
we start with the first tab and use =Ctrl+F= as we've seen before,
|
|
|
|
|
and then type ``free software''.
|
|
|
|
|
If we /do not/ find a match,
|
|
|
|
|
we're ready to copy the URL.
|
|
|
|
|
Hitting =Ctrl+L= will take us to the location bar and highlight the URL.
|
|
|
|
|
We can then hit =Ctrl+C= to copy the URL to the clipboard.
|
2019-03-20 23:49:14 -04:00
|
|
|
|
=Alt+Tab= to go to our text editor.
|
2019-03-08 22:38:21 -05:00
|
|
|
|
We then paste with =Ctrl+V= and hit return to insert a newline.
|
|
|
|
|
We can then go back to the web browser by hitting =Alt+Tab= again.
|
2019-03-20 00:01:10 -04:00
|
|
|
|
We then close the tab with =Ctrl+W=.
|
2019-03-08 22:38:21 -05:00
|
|
|
|
|
|
|
|
|
Repeat, and we're done all the same as before.
|
|
|
|
|
As a bonus,
|
|
|
|
|
save with =Ctrl+S=.
|
|
|
|
|
|
|
|
|
|
What's interesting about this approach is that we didn't have to use the
|
|
|
|
|
mouse at all,
|
|
|
|
|
unless maybe you used it to highlight the URL in the email.
|
|
|
|
|
You could get into quite the rhythm with this approach,
|
|
|
|
|
and your hands never have to leave the keyboard.
|
|
|
|
|
This is a bit of a faster,
|
|
|
|
|
more efficient way to convey our thoughts to the machine,
|
|
|
|
|
right?
|
|
|
|
|
We don't have to seek out our actions each time in the GUI---the
|
|
|
|
|
operations are always at our fingertips,
|
|
|
|
|
literally.
|
|
|
|
|
|
2019-03-25 12:53:44 -04:00
|
|
|
|
*** REHEARSED GUIs of a Feather :B_fullframe:
|
2019-03-08 22:38:21 -05:00
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: fullframe
|
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
#+BEAMER: \fullslidetext
|
|
|
|
|
Same Keybindings Across (Most) GUIs!
|
|
|
|
|
|
|
|
|
|
#+BEAMER: \fullsubtext
|
|
|
|
|
Browser, Editor, Window Manager, OS, \ldots
|
|
|
|
|
|
|
|
|
|
**** Notes :B_noteNH:
|
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: noteNH
|
2019-03-20 23:49:14 -04:00
|
|
|
|
:DURATION: 00:00:35
|
2019-03-08 22:38:21 -05:00
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
Another powerful benefit of this approach is---these
|
|
|
|
|
/same exact keybindings work across most GUIs/!
|
2019-03-12 00:35:14 -04:00
|
|
|
|
If we switch out Icecat here with nearly any other major web browser,
|
2019-03-08 22:38:21 -05:00
|
|
|
|
and switch out gedit with many other text editors or even word processors,
|
|
|
|
|
this will work all the same!
|
|
|
|
|
|
|
|
|
|
If you use Windows instead of GNU/Linux---which
|
2019-03-12 00:35:14 -04:00
|
|
|
|
I strongly discourage, but if you do---then
|
2019-03-08 22:38:21 -05:00
|
|
|
|
it'll work the same.
|
|
|
|
|
|
2019-03-20 23:49:14 -04:00
|
|
|
|
Certain keybindings are ubiquitous---if
|
|
|
|
|
you hit =Ctrl+F= in most GUI programs that support searching,
|
|
|
|
|
you'll get some sort of context-specific search.
|
|
|
|
|
|
2019-03-08 22:38:21 -05:00
|
|
|
|
Let's look at those keybindings a bit more concisely,
|
2019-03-20 00:01:10 -04:00
|
|
|
|
since that last slide was a bit of a mess.
|
2019-03-08 22:38:21 -05:00
|
|
|
|
|
2019-03-25 12:53:44 -04:00
|
|
|
|
*** REHEARSED Macro-Like Keyboard Instructions :B_fullframe:
|
2019-03-08 22:38:21 -05:00
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: fullframe
|
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
#+BEAMER: \begingroup\fullslidetext
|
|
|
|
|
Macro-Like
|
|
|
|
|
#+BEAMER: \endgroup\subskip
|
|
|
|
|
|
|
|
|
|
#+BEGIN_SRC
|
|
|
|
|
Ctrl+T ``https://...'' <N times>
|
|
|
|
|
|
|
|
|
|
Ctrl+F ``free sofware''
|
|
|
|
|
[ Ctrl+L Ctrl+C Alt+Tab Ctrl+V RET Alt+Tab ]
|
|
|
|
|
Ctrl+W
|
|
|
|
|
<N times>
|
|
|
|
|
#+END_SRC
|
|
|
|
|
|
|
|
|
|
- <2> /Requires visual inspection/ for conditional
|
|
|
|
|
- <2> Still manual and tedious---what if there were 1000 URLs?
|
|
|
|
|
|
|
|
|
|
**** Notes :B_noteNH:
|
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: noteNH
|
2019-03-20 00:01:10 -04:00
|
|
|
|
:DURATION: 00:01:00
|
2019-03-08 22:38:21 -05:00
|
|
|
|
:END:
|
|
|
|
|
|
2019-03-12 00:35:14 -04:00
|
|
|
|
If we type out the keybindings like this,
|
2019-03-08 22:38:21 -05:00
|
|
|
|
it looks a bit more like instructions for the machine,
|
|
|
|
|
doesn't it?
|
|
|
|
|
Some of you may be familiar with macros---with
|
|
|
|
|
the ability to record keypresses and play them back later.
|
2019-03-12 00:35:14 -04:00
|
|
|
|
If we were able to do that,
|
2019-03-20 00:01:10 -04:00
|
|
|
|
could we automate this task away?
|
2019-03-08 22:38:21 -05:00
|
|
|
|
|
2019-03-20 00:01:10 -04:00
|
|
|
|
Unfortunately...no.
|
2019-03-08 22:38:21 -05:00
|
|
|
|
At least,
|
|
|
|
|
not with the tools we're using right now.
|
|
|
|
|
Why is that?
|
|
|
|
|
|
|
|
|
|
Well,
|
|
|
|
|
for one,
|
|
|
|
|
it requires visual inspection to determine whether or not a match has
|
|
|
|
|
occurred.
|
|
|
|
|
That drives conditional logic---that
|
|
|
|
|
bracketed part there.
|
|
|
|
|
We also need to know how many times to repeat,
|
|
|
|
|
which requires that we either count or watch the progress.
|
2019-03-20 00:01:10 -04:00
|
|
|
|
And we need to be able to inspect the email for URLs and copy them into the
|
2019-03-08 22:38:21 -05:00
|
|
|
|
web browser.
|
|
|
|
|
|
2019-03-12 00:35:14 -04:00
|
|
|
|
This also scales really poorly.
|
2019-03-08 22:38:21 -05:00
|
|
|
|
While using the keyboard is certainly faster than using the mouse,
|
|
|
|
|
we're only dealing with a small set of URLs here.
|
|
|
|
|
What if I gave you 100 of them?
|
|
|
|
|
1000?
|
|
|
|
|
More?
|
|
|
|
|
Suddenly this doesn't feel like a very efficient way to convey our intent to
|
|
|
|
|
the machine.
|
|
|
|
|
I don't wish that suffering upon anyone.
|
|
|
|
|
|
2019-03-12 00:35:14 -04:00
|
|
|
|
To get around that,
|
2019-03-08 22:38:21 -05:00
|
|
|
|
we need to change how we think about our computing a bit.
|
|
|
|
|
|
2019-03-25 12:53:44 -04:00
|
|
|
|
** REHEARSED A New Perspective [15/15]
|
2019-03-10 23:18:05 -04:00
|
|
|
|
*** Perspective Topics [13/13] :noexport:
|
2019-03-10 22:38:26 -04:00
|
|
|
|
- [X] What if I could walk away and get a coffee, play with the kids,
|
|
|
|
|
come back and have it done for me?
|
|
|
|
|
- Emphasize how users familiar with the tools I haven't yet mentioned
|
|
|
|
|
may know how to do this, but people who don't know of Unix tools
|
|
|
|
|
will still be thinking in terms of the UI, trying to figure out how
|
|
|
|
|
to automate it.
|
|
|
|
|
- [X] Right-click on page and inspect an element.
|
|
|
|
|
- Briefly mention DOM and how this represents the current state of
|
|
|
|
|
the page.
|
|
|
|
|
- Maybe show how it can be modified to illustrate the structure better.
|
|
|
|
|
- But I don't want to get too deep into this.
|
|
|
|
|
- [X] Right-click on page and View Source.
|
|
|
|
|
- Explain how this is like the source code to the webpage (HTML).
|
|
|
|
|
- [X] Copy text into editor, =C-f=.
|
|
|
|
|
- We can still find the text /using the same keybinding/.
|
|
|
|
|
- [X] Save HTML file.
|
|
|
|
|
- [X] Demonstrate opening it in a web browser vs. an editor.
|
|
|
|
|
- Same exact document, different ways of rendering it.
|
|
|
|
|
- [X] Open a terminal [4/4]
|
|
|
|
|
- [X] Type =wget URL=
|
|
|
|
|
- [X] Show how it downloaded a file
|
|
|
|
|
- [X] Open it in a text editor, show how the text is there. =C-f=.
|
|
|
|
|
- We eliminated the web browser
|
|
|
|
|
- [X] =grep word file.html=
|
|
|
|
|
- We have just eliminated =Ctrl+F= in the editor.
|
|
|
|
|
- [X] But it's too verbose, so show =grep -q && echo=.
|
|
|
|
|
- Talk about echo and exit status briefly, but don't go into exit codes.
|
|
|
|
|
- [X] The goal is to output a URL to a file
|
|
|
|
|
- Introduce redirection (overwrite and append)
|
|
|
|
|
- [X] Now we have duplication: URL in two places
|
|
|
|
|
- Introduce variables
|
|
|
|
|
- [X] Eliminate the temporary file entirely using a pipe
|
2019-03-10 23:18:05 -04:00
|
|
|
|
- [X] Alias =fetch-url= for =wget -qO-=.
|
2019-03-10 22:38:26 -04:00
|
|
|
|
- [X] Introduce the Unix philosophy
|
|
|
|
|
|
2019-03-25 12:53:44 -04:00
|
|
|
|
*** REHEARSED Secrets? :B_fullframe:
|
2019-03-10 22:38:26 -04:00
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: fullframe
|
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
# Slide intentionally left blank
|
|
|
|
|
|
|
|
|
|
**** Notes :B_noteNH:
|
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: noteNH
|
2019-03-21 22:45:40 -04:00
|
|
|
|
:DURATION: 00:00:55
|
2019-03-10 22:38:26 -04:00
|
|
|
|
:END:
|
|
|
|
|
|
2019-03-21 22:45:40 -04:00
|
|
|
|
So what if I told you that I could go grab some coffee and play with my kids
|
|
|
|
|
and come back later to a list that has been generated for me by an
|
|
|
|
|
automated process?
|
2019-03-10 22:38:26 -04:00
|
|
|
|
And what if I told you that it'd only take a minute or two to for me to
|
|
|
|
|
create this process?
|
2019-03-21 22:45:40 -04:00
|
|
|
|
And that you don't need to be a programmer to do it?
|
|
|
|
|
|
|
|
|
|
Well if I told you that then you'd be pretty pissed at me for sending you
|
|
|
|
|
1000 URLs.
|
|
|
|
|
So I would never do that.
|
2019-03-10 22:38:26 -04:00
|
|
|
|
|
|
|
|
|
This is where the whole concept of ``wizardry'' comes in.
|
|
|
|
|
Some of you are sitting in the audience or watching this remotely rolling
|
|
|
|
|
your eyes thinking ``oh this guy thinks he's so sweet'',
|
|
|
|
|
because the answer is obvious to you.
|
|
|
|
|
But to those of you who are confined to the toolset that I just
|
|
|
|
|
demonstrated...it's
|
2019-03-12 00:35:14 -04:00
|
|
|
|
/not/ going to be obvious.
|
2019-03-10 22:38:26 -04:00
|
|
|
|
|
|
|
|
|
The problem is that there is a whole world and way of computing that is
|
|
|
|
|
hidden from most users.
|
|
|
|
|
And it's not hidden because it's a secret.
|
|
|
|
|
It's because modern interfaces have come to completely mask it or provide
|
2019-03-21 22:45:40 -04:00
|
|
|
|
alternatives to it that happen to be ``good enough'' for a particular use
|
|
|
|
|
case.
|
2019-03-10 22:38:26 -04:00
|
|
|
|
|
2019-03-25 12:53:44 -04:00
|
|
|
|
But ``good enough'' is only good enough until it's not,
|
|
|
|
|
until you need to do something /outside/ of that preconceived workflow.
|
2019-03-10 22:38:26 -04:00
|
|
|
|
|
2019-03-25 12:53:44 -04:00
|
|
|
|
*** REHEARSED Lifting the Curtain :B_frame:
|
2019-03-10 22:38:26 -04:00
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: frame
|
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
**** Columns :B_columns:
|
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: columns
|
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
***** Left :B_column:
|
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: column
|
|
|
|
|
:BEAMER_col: 0.30
|
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
[[file:images/ff-view-source-menu.png]]
|
|
|
|
|
|
|
|
|
|
***** Right :B_column:
|
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: column
|
|
|
|
|
:BEAMER_col: 0.70
|
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
[[file:images/ff-view-source.png]]
|
|
|
|
|
|
|
|
|
|
**** Notes :B_noteNH:
|
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: noteNH
|
2019-03-12 00:35:14 -04:00
|
|
|
|
:DURATION: 00:00:35
|
2019-03-10 22:38:26 -04:00
|
|
|
|
:END:
|
|
|
|
|
|
2019-03-20 22:19:23 -04:00
|
|
|
|
Let's lift the curtain,
|
|
|
|
|
so to speak,
|
|
|
|
|
on what's really going on in the web browser.
|
|
|
|
|
|
|
|
|
|
Right-click on the webpage and select "View Page Source" from the
|
|
|
|
|
context menu.
|
|
|
|
|
You get a new tab containing the raw code behind the document.
|
2019-03-10 22:38:26 -04:00
|
|
|
|
Most of it, anyway.
|
|
|
|
|
It is a document language called HTML.
|
|
|
|
|
And as you may have noticed,
|
|
|
|
|
it's plain text.
|
|
|
|
|
Structured, but plain, text.
|
|
|
|
|
|
|
|
|
|
And as you can see,
|
2019-03-12 00:35:14 -04:00
|
|
|
|
if we hit =Ctrl+F=,
|
2019-03-10 22:38:26 -04:00
|
|
|
|
``free software'' is there all the same.
|
|
|
|
|
We don't need to view the webpage with all its fancy formatting.
|
|
|
|
|
For the problem we're trying to solve,
|
2019-03-12 00:35:14 -04:00
|
|
|
|
the graphical representation provides little benefit.
|
2019-03-10 22:38:26 -04:00
|
|
|
|
|
2019-03-25 12:53:44 -04:00
|
|
|
|
*** REHEARSED Text :B_fullframe:
|
2019-03-10 22:38:26 -04:00
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: fullframe
|
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
#+BEAMER: \fullslidetext
|
|
|
|
|
Text.
|
|
|
|
|
|
|
|
|
|
**** Notes :B_noteNH:
|
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: noteNH
|
2019-03-12 00:35:14 -04:00
|
|
|
|
:DURATION: 00:00:35
|
2019-03-10 22:38:26 -04:00
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
As we're about to see,
|
|
|
|
|
this simple fact---that
|
|
|
|
|
that the webpage is represented by plain text---opens
|
|
|
|
|
up a whole new world to us.
|
|
|
|
|
We have stripped away all the complex visual GUI stuff and we're left with
|
|
|
|
|
the raw substance of the page which still contains the information that we
|
|
|
|
|
are looking for.
|
|
|
|
|
|
|
|
|
|
But we're still within the web browser.
|
|
|
|
|
|
|
|
|
|
We don't have to be.
|
|
|
|
|
We can copy all of that text and paste it into our editor.
|
2019-03-12 00:35:14 -04:00
|
|
|
|
=Ctrl+A Ctrl+C Alt-Tab Ctrl+V=.
|
2019-03-10 22:38:26 -04:00
|
|
|
|
And sure enough,
|
|
|
|
|
search works all the same.
|
|
|
|
|
=Ctrl+F= and we can still find ``free software''.
|
|
|
|
|
Completely different program,
|
|
|
|
|
and we can still find the text using the same keybinding.
|
|
|
|
|
|
2019-03-25 12:53:44 -04:00
|
|
|
|
*** REHEARSED Text is a Universal Interface :B_fullframe:
|
2019-03-10 22:38:26 -04:00
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: fullframe
|
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
#+BEAMER: \fullslidetext
|
|
|
|
|
Text is a Universal Interface
|
|
|
|
|
|
|
|
|
|
**** Notes :B_noteNH:
|
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: noteNH
|
2019-03-25 12:53:44 -04:00
|
|
|
|
:DURATION: 00:01:10
|
2019-03-10 22:38:26 -04:00
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
Text is a universal interface.
|
|
|
|
|
And what I mean by that is---you
|
|
|
|
|
don't need any special tools to work with it.
|
|
|
|
|
You can view it in your web browser.
|
|
|
|
|
You can view it in your text editor.
|
|
|
|
|
You can paste it in a text message.
|
2019-03-20 00:01:10 -04:00
|
|
|
|
You can print it in a book.
|
2019-03-10 22:38:26 -04:00
|
|
|
|
You can write it down on a paper and type it back into your computer.
|
|
|
|
|
|
|
|
|
|
Text is how we communicate with one-another as human beings.
|
|
|
|
|
|
|
|
|
|
Let's save this HTML as a file,
|
|
|
|
|
=speakers.html=.
|
|
|
|
|
|
|
|
|
|
If we opened this file,
|
|
|
|
|
it would open in our web browser and we would see the same webpage,
|
|
|
|
|
although it would look a bit different since a lot of the styling is
|
2019-03-21 22:45:40 -04:00
|
|
|
|
stored outside of this HTML file.
|
2019-03-10 22:38:26 -04:00
|
|
|
|
|
|
|
|
|
But if again we opened this HTML file in our text editor,
|
|
|
|
|
you would see that same plain text HTML as before;
|
2019-03-12 00:35:14 -04:00
|
|
|
|
one program just chooses to render it differently than another.
|
2019-03-10 22:38:26 -04:00
|
|
|
|
|
|
|
|
|
Even though we can view the HTML in our text editor,
|
2019-03-12 00:35:14 -04:00
|
|
|
|
we haven't yet eliminated the web browser;
|
2019-03-10 22:38:26 -04:00
|
|
|
|
we still need it to navigate to the webpage and view its source.
|
|
|
|
|
But if that's all we're using the web browser for,
|
|
|
|
|
then it's one hell of an inefficient way of telling the computer that we
|
|
|
|
|
just want the HTML document at a certain URL.
|
|
|
|
|
|
|
|
|
|
Up until this point,
|
|
|
|
|
the keyboard has been used as a secondary interface---as
|
|
|
|
|
an /alternative/ to something.
|
|
|
|
|
Now we're going to venture into a world where it is /the/ interface.
|
|
|
|
|
|
2019-03-25 12:53:44 -04:00
|
|
|
|
*** REHEARSED The Shell Command Prompt :B_frame:
|
2019-03-10 22:38:26 -04:00
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: frame
|
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
#+BEGIN_SRC sh
|
|
|
|
|
mikegerwitz@lp2019-laptop:~$
|
|
|
|
|
# ^ user ^ host ^ working directory (home)
|
|
|
|
|
#+END_SRC
|
|
|
|
|
|
|
|
|
|
#+BEAMER: \begin{uncoverenv}<2->
|
|
|
|
|
#+BEAMER: \subskip
|
|
|
|
|
This presentation will show:
|
|
|
|
|
#+BEAMER: \medskip
|
|
|
|
|
|
|
|
|
|
#+BEGIN_SRC
|
|
|
|
|
$ command
|
|
|
|
|
output line 1
|
|
|
|
|
output line 2
|
|
|
|
|
...
|
|
|
|
|
output line N
|
|
|
|
|
#+END_SRC
|
|
|
|
|
#+BEAMER: \end{uncoverenv}
|
|
|
|
|
|
|
|
|
|
**** Notes :B_noteNH:
|
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: noteNH
|
2019-03-21 22:45:40 -04:00
|
|
|
|
:DURATION: 00:00:55
|
2019-03-10 22:38:26 -04:00
|
|
|
|
:END:
|
|
|
|
|
|
2019-03-20 00:01:10 -04:00
|
|
|
|
If you open a VTE,
|
|
|
|
|
or virtual terminal emulator,
|
2019-03-10 22:38:26 -04:00
|
|
|
|
you will be greeted with a curious string of characters.
|
|
|
|
|
This is a /command prompt/.
|
|
|
|
|
|
|
|
|
|
The program that is prompting you for a command is called the /shell/.
|
|
|
|
|
The GNU shell is =bash=,
|
|
|
|
|
which is the default on most GNU/Linux systems.
|
2019-03-21 22:45:40 -04:00
|
|
|
|
|
|
|
|
|
Since we're talking about freedom in your computing,
|
|
|
|
|
I hope that you'll try bash on an operating system that respects your
|
|
|
|
|
freedom to use, study, modify, and share its software,
|
|
|
|
|
like a GNU/Linux distribution.
|
|
|
|
|
This laptop here runs Trisquel.
|
|
|
|
|
|
|
|
|
|
Bash is also the default on Mac OSX,
|
|
|
|
|
but the rest of the operating system is non-free;
|
|
|
|
|
you have no freedom there.
|
2019-03-10 22:38:26 -04:00
|
|
|
|
And Windows now has something they call
|
|
|
|
|
``Bash on Ubuntu on Windows'',
|
2019-03-25 12:53:44 -04:00
|
|
|
|
which just is GNU running atop of the proprietary Windows kernel.
|
2019-03-21 22:45:40 -04:00
|
|
|
|
You can do better that that!
|
2019-03-10 22:38:26 -04:00
|
|
|
|
|
2019-03-21 22:45:40 -04:00
|
|
|
|
When I present commands here,
|
|
|
|
|
the command line we are executing is prefixed with a dollar sign,
|
|
|
|
|
and the output immediately follows it,
|
|
|
|
|
like so.
|
2019-03-10 22:38:26 -04:00
|
|
|
|
|
2019-03-25 12:53:44 -04:00
|
|
|
|
*** REHEARSED Eliminating the Web Browser :B_frame:
|
2019-03-10 22:38:26 -04:00
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: frame
|
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
#+BEGIN_SRC sh
|
|
|
|
|
$ wget https://libreplanet.org/2019/speakers/
|
|
|
|
|
#+END_SRC
|
|
|
|
|
|
2019-03-12 00:35:14 -04:00
|
|
|
|
#+BEAMER: \begin{uncoverenv}<2->
|
2019-03-10 22:38:26 -04:00
|
|
|
|
#+BEGIN_SRC sh
|
|
|
|
|
--2019-03-24 00:00:00-- https://libreplanet.org/2019/speakers/
|
|
|
|
|
Resolving libreplanet.org (libreplanet.org)... 209.51.188.248
|
|
|
|
|
Connecting to libreplanet.org (libreplanet.org)|209.51.188.248|:443... connected.
|
|
|
|
|
HTTP request sent, awaiting response... 200 OK
|
|
|
|
|
Length: unspecified [text/html]
|
|
|
|
|
Saving to: ‘index.html’
|
|
|
|
|
...
|
|
|
|
|
2019-03-24 00:00:00 (1.78 MB/s) - ‘index.html’ saved [67789]
|
|
|
|
|
#+END_SRC
|
|
|
|
|
#+BEAMER: \end{uncoverenv}
|
|
|
|
|
|
|
|
|
|
#+BEAMER: \begin{uncoverenv}<3>
|
|
|
|
|
#+BEGIN_SRC sh
|
|
|
|
|
$ wget -O speakers.html \
|
|
|
|
|
https://libreplanet.org/2019/speakers/
|
|
|
|
|
#+END_SRC
|
|
|
|
|
#+BEAMER: \end{uncoverenv}
|
|
|
|
|
|
|
|
|
|
**** Notes :B_noteNH:
|
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: noteNH
|
2019-03-21 22:45:40 -04:00
|
|
|
|
:DURATION: 00:01:05
|
2019-03-10 22:38:26 -04:00
|
|
|
|
:END:
|
|
|
|
|
|
2019-03-12 00:35:14 -04:00
|
|
|
|
Alright!
|
2019-03-10 22:38:26 -04:00
|
|
|
|
The goal is to retrieve the HTML file at a given URL.
|
2019-03-21 22:45:40 -04:00
|
|
|
|
|
2019-03-10 22:38:26 -04:00
|
|
|
|
GNU/Linux distributions usually come with GNU =wget=,
|
|
|
|
|
which does precisely that.
|
|
|
|
|
To invoke it,
|
|
|
|
|
we type the name of the command,
|
|
|
|
|
followed by a space,
|
|
|
|
|
followed by the URL we wish to retrieve,
|
|
|
|
|
and then hit enter.
|
|
|
|
|
|
|
|
|
|
What follows is quite a bit of text.
|
|
|
|
|
The details aren't particularly important as long as it's successful,
|
|
|
|
|
but notice that it says it saved to =index.html=.
|
2019-03-12 00:35:14 -04:00
|
|
|
|
That's not intuitive to those who don't understand why that name was used.
|
2019-03-10 22:38:26 -04:00
|
|
|
|
|
|
|
|
|
So let's tell =wget= what file we want to output to.
|
|
|
|
|
We do this with the =O= option,
|
|
|
|
|
like so.
|
|
|
|
|
It takes a single argument,
|
|
|
|
|
which is the name of the output file.
|
|
|
|
|
The backslash here allows us to continue the command onto the next line;
|
|
|
|
|
otherwise, a newline tells the shell to execute the command.
|
|
|
|
|
|
|
|
|
|
So remember previously that we manually created =speakers.html= by viewing
|
2019-03-21 22:45:40 -04:00
|
|
|
|
the source of the webpage in a web browser and saving it.
|
2019-03-10 22:38:26 -04:00
|
|
|
|
If we open this file,
|
2019-03-21 22:45:40 -04:00
|
|
|
|
we'll find that it contains /exactly the same text/ as when we manually
|
|
|
|
|
did it,
|
|
|
|
|
and we never had to open a web browser.
|
2019-03-12 00:35:14 -04:00
|
|
|
|
And we can search it all the same as before for ``free software''.
|
2019-03-10 22:38:26 -04:00
|
|
|
|
|
2019-03-25 12:53:44 -04:00
|
|
|
|
*** REHEARSED Browser vs. =wget= Comparison :B_frame:
|
2019-03-10 22:38:26 -04:00
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: frame
|
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
#+BEGIN_SRC
|
|
|
|
|
Ctrl+L ``https://libreplanet.org/2019/speakers/''
|
|
|
|
|
#+END_SRC
|
|
|
|
|
|
|
|
|
|
#+BEAMER: \subskip
|
|
|
|
|
|
|
|
|
|
#+BEGIN_SRC sh
|
|
|
|
|
$ wget https://libreplanet.org/2019/speakers/
|
|
|
|
|
#+END_SRC
|
|
|
|
|
|
|
|
|
|
**** Notes :B_noteNH:
|
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: noteNH
|
2019-03-21 22:45:40 -04:00
|
|
|
|
:DURATION: 00:00:20
|
2019-03-10 22:38:26 -04:00
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
This is a very different means of interacting with the computer,
|
|
|
|
|
but if we compare this with the keyboard shortcut used previously,
|
|
|
|
|
they are very similar.
|
|
|
|
|
Not so scary,
|
|
|
|
|
right?
|
|
|
|
|
It's hard to imagine a more direct line of communication with the computer
|
|
|
|
|
for downloading a webpage,
|
|
|
|
|
short of reading your mind.
|
|
|
|
|
|
2019-03-25 12:53:44 -04:00
|
|
|
|
*** REHEARSED Finding Text on the Command Line :B_frame:
|
2019-03-10 22:38:26 -04:00
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: frame
|
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
#+BEGIN_SRC sh
|
|
|
|
|
$ grep 'free software' speakers.html
|
|
|
|
|
#+END_SRC
|
|
|
|
|
|
|
|
|
|
#+BEAMER: \begin{uncoverenv}<2->
|
|
|
|
|
\vdots
|
|
|
|
|
#+BEGIN_SRC
|
|
|
|
|
<p>Mike Gerwitz is a free software hacker and activist with a focus on
|
|
|
|
|
exclusively free software. Mike spends most of his free time with his
|
|
|
|
|
#+END_SRC
|
|
|
|
|
\vdots
|
|
|
|
|
#+BEAMER: \end{uncoverenv}
|
|
|
|
|
|
|
|
|
|
**** Notes :B_noteNH:
|
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: noteNH
|
2019-03-12 00:35:14 -04:00
|
|
|
|
:DURATION: 00:01:00
|
2019-03-10 22:38:26 -04:00
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
Not having to open a web browser is nice,
|
2019-03-20 00:01:10 -04:00
|
|
|
|
but having to open the downloaded HTML file just to search it is a bit of
|
|
|
|
|
a pain;
|
|
|
|
|
is there a command that can help us there too?
|
2019-03-10 22:38:26 -04:00
|
|
|
|
|
|
|
|
|
We want to know whether a page contains the term ``free software''.
|
|
|
|
|
For that we use a tool called =grep=.
|
|
|
|
|
|
|
|
|
|
The first argument to =grep= is the search string,
|
|
|
|
|
and the remaining arguments---just one here---tell it where it should
|
|
|
|
|
search.
|
2019-03-21 22:45:40 -04:00
|
|
|
|
The first argument to =grep= is quoted because it contains a space.
|
2019-03-10 22:38:26 -04:00
|
|
|
|
|
|
|
|
|
You'll get a bunch of output;
|
|
|
|
|
I just included a small snippet here.
|
|
|
|
|
But notice how it happens to include exactly the text we were looking at in
|
|
|
|
|
the web browser.
|
|
|
|
|
|
|
|
|
|
And with that we have replicated =Ctrl+F=.
|
|
|
|
|
But did we do a good job conveying our thoughts to the machine?
|
|
|
|
|
|
2019-03-20 00:01:10 -04:00
|
|
|
|
We just wanted to know whether the page /contains/ the phrase,
|
|
|
|
|
but we don't care to see it!
|
|
|
|
|
So while we have efficiently /conveyed/ a search string,
|
|
|
|
|
we didn't receive an efficient /reply/---it's
|
2019-03-12 00:35:14 -04:00
|
|
|
|
information overload.
|
2019-03-10 22:38:26 -04:00
|
|
|
|
|
2019-03-25 12:53:44 -04:00
|
|
|
|
*** REHEARSED A More Gentle Reply :B_frame:
|
2019-03-10 22:38:26 -04:00
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: frame
|
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
#+BEGIN_SRC sh
|
|
|
|
|
$ grep --quiet 'free software' speakers.html && echo yes
|
|
|
|
|
|
|
|
|
|
yes
|
|
|
|
|
#+END_SRC
|
|
|
|
|
|
|
|
|
|
#+BEAMER: \subskip
|
|
|
|
|
|
|
|
|
|
#+BEAMER: \begin{onlyenv}<2>
|
|
|
|
|
#+BEGIN_SRC sh
|
|
|
|
|
$ echo 'Hello, world!'
|
|
|
|
|
|
|
|
|
|
Hello, world!
|
|
|
|
|
#+END_SRC
|
|
|
|
|
#+BEAMER: \end{onlyenv}
|
|
|
|
|
|
|
|
|
|
#+BEAMER: \begin{onlyenv}<3>
|
|
|
|
|
#+BEGIN_SRC sh
|
|
|
|
|
$ grep --quiet 'open source' speakers.html || echo no
|
|
|
|
|
|
|
|
|
|
no
|
|
|
|
|
#+END_SRC
|
|
|
|
|
#+BEAMER: \end{onlyenv}
|
|
|
|
|
|
|
|
|
|
**** Notes :B_noteNH:
|
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: noteNH
|
2019-03-12 00:35:14 -04:00
|
|
|
|
:DURATION: 00:01:00
|
2019-03-10 22:38:26 -04:00
|
|
|
|
:END:
|
|
|
|
|
|
2019-03-12 00:35:14 -04:00
|
|
|
|
First we tell =grep= to modify its behavior with the =quiet= flag.
|
|
|
|
|
You can also use the short form,
|
|
|
|
|
which is just =-q=.
|
2019-03-10 22:38:26 -04:00
|
|
|
|
Rather than outputting results,
|
2019-03-12 00:35:14 -04:00
|
|
|
|
=grep= will exit silently and it will instead return a status to the shell
|
|
|
|
|
that says whether or not the search failed.
|
2019-03-10 22:38:26 -04:00
|
|
|
|
|
|
|
|
|
POSIX-like shells,
|
|
|
|
|
like Bash,
|
2019-03-20 00:01:10 -04:00
|
|
|
|
offer the ability to say ``run this next command if the previous
|
|
|
|
|
succeeds'',
|
|
|
|
|
and this is done by putting two ampersands between the commands.
|
2019-03-10 22:38:26 -04:00
|
|
|
|
|
|
|
|
|
The command to run if =grep= succeeds in finding a match is =echo=.
|
|
|
|
|
All echo does is takes its arguments and spits them right back out again as
|
|
|
|
|
output.
|
|
|
|
|
So this essentially states:
|
|
|
|
|
``search for `free software' in =speakers.html= and output `yes' if it is
|
|
|
|
|
found''.
|
|
|
|
|
|
|
|
|
|
Since =echo= is its own command,
|
|
|
|
|
it also works by itself.
|
|
|
|
|
Here's the classic ``hello, world'' program in shell.
|
|
|
|
|
|
|
|
|
|
But if you recall our research task,
|
|
|
|
|
it was to search for pages that do /not/ contain the term ``free
|
|
|
|
|
software''.
|
|
|
|
|
We can do that too,
|
2019-03-20 00:01:10 -04:00
|
|
|
|
by using two pipes in place of two ampersands.
|
2019-03-10 22:38:26 -04:00
|
|
|
|
|
2019-03-25 12:53:44 -04:00
|
|
|
|
*** REHEARSED Writing to Files (Redirection) :B_frame:
|
2019-03-10 22:38:26 -04:00
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: frame
|
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
- Commands write to standard out (stdout) by default
|
|
|
|
|
- /Output redirection/ writes somewhere else
|
|
|
|
|
|
|
|
|
|
#+BEAMER: \subskip
|
|
|
|
|
|
|
|
|
|
#+BEGIN_SRC sh
|
|
|
|
|
# overwrites each time
|
|
|
|
|
$ echo 'Hello, world!' > hello.txt
|
|
|
|
|
$ echo 'Hello again, world!' > hello.txt
|
|
|
|
|
|
|
|
|
|
# appends (echo adds a newline)
|
|
|
|
|
$ echo 'First line' >> results.txt
|
|
|
|
|
$ echo 'Second line' >> results.txt
|
|
|
|
|
#+END_SRC
|
|
|
|
|
|
|
|
|
|
**** Notes :B_noteNH:
|
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: noteNH
|
2019-03-12 00:35:14 -04:00
|
|
|
|
:DURATION: 00:00:55
|
2019-03-10 22:38:26 -04:00
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
Alright, we're well on our way now!
|
|
|
|
|
But we still haven't gotten rid of that damn text editor,
|
|
|
|
|
because we need to save a list of URLs to a file to hold our final
|
|
|
|
|
results!
|
|
|
|
|
|
|
|
|
|
Well as it so happens,
|
|
|
|
|
writing to a file is such a common operation that it's built right into
|
|
|
|
|
the shell.
|
|
|
|
|
We use a feature called /redirection/.
|
|
|
|
|
|
|
|
|
|
There are two types of output redirection.
|
|
|
|
|
If you place a single greater-than symbol followed by a filename after a
|
|
|
|
|
command,
|
|
|
|
|
then the output of that command will replace anything already in the
|
|
|
|
|
file.
|
2019-03-12 00:35:14 -04:00
|
|
|
|
So the result of the first two commands will be a =hello.txt= that contains
|
2019-03-10 22:38:26 -04:00
|
|
|
|
only a single line:
|
|
|
|
|
``Hello again, world!''.
|
|
|
|
|
|
|
|
|
|
The second type,
|
|
|
|
|
which uses /two/ greater-than symbols,
|
2019-03-12 00:35:14 -04:00
|
|
|
|
/appends/ to the file.
|
2019-03-10 22:38:26 -04:00
|
|
|
|
=echo= by default adds a newline,
|
|
|
|
|
so the result of the second two commands is a =results.txt= containing two
|
|
|
|
|
lines,
|
|
|
|
|
``First line'' and ``Second line'' respectively.
|
|
|
|
|
If the file doesn't yet exist,
|
2019-03-21 22:45:40 -04:00
|
|
|
|
it will be created.
|
2019-03-10 22:38:26 -04:00
|
|
|
|
|
2019-03-25 12:53:44 -04:00
|
|
|
|
*** REHEARSED Starting Our List :B_fullframe:
|
2019-03-10 22:38:26 -04:00
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: fullframe
|
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
#+BEGIN_SRC sh
|
|
|
|
|
|
2019-03-12 00:35:14 -04:00
|
|
|
|
|
|
|
|
|
|
2019-03-10 22:38:26 -04:00
|
|
|
|
$ wget --quiet -O speakers.html \
|
|
|
|
|
https://libreplanet.org/2019/speakers/ \
|
|
|
|
|
&& grep --quiet 'free software' speakers.html \
|
|
|
|
|
|| echo https://libreplanet.org/2019/speakers/ \
|
|
|
|
|
>> results.txt
|
|
|
|
|
#+END_SRC
|
|
|
|
|
|
|
|
|
|
**** Notes :B_noteNH:
|
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: noteNH
|
2019-03-20 00:01:10 -04:00
|
|
|
|
:DURATION: 00:01:00
|
2019-03-10 22:38:26 -04:00
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
Take a look at that for a moment.
|
|
|
|
|
/<pause ~5s>/
|
2019-03-21 22:45:40 -04:00
|
|
|
|
Can anyone tell me in just one sentence what the result of this command line
|
|
|
|
|
will be?
|
2019-03-10 22:38:26 -04:00
|
|
|
|
/<pause ~5s>/
|
|
|
|
|
/<react appropriately>/
|
|
|
|
|
|
|
|
|
|
As exciting as it is to start to bring these things together,
|
|
|
|
|
the result is pretty anti-climatic---we
|
|
|
|
|
know that =speakers.html= /does/ contain the string ``free software'',
|
|
|
|
|
and so the result is that =results.txt= contains /nothing/!
|
|
|
|
|
In fact,
|
|
|
|
|
if =results.txt= didn't exist yet,
|
|
|
|
|
it still wouldn't even exist.
|
|
|
|
|
|
|
|
|
|
At this point,
|
|
|
|
|
we have successfully eliminated both the web browser and text editor.
|
|
|
|
|
But this is a hefty command to have to modify each time we want to try a
|
|
|
|
|
different URL.
|
|
|
|
|
|
2019-03-25 12:53:44 -04:00
|
|
|
|
*** REHEARSED Command Refactoring :B_fullframe:
|
2019-03-10 22:38:26 -04:00
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: fullframe
|
|
|
|
|
:END:
|
|
|
|
|
|
2019-03-10 23:18:05 -04:00
|
|
|
|
#+BEAMER: \begin{onlyenv}<6>
|
2019-03-10 22:38:26 -04:00
|
|
|
|
#+BEGIN_SRC sh
|
|
|
|
|
|
2019-03-20 00:01:10 -04:00
|
|
|
|
# original:
|
2019-03-12 00:35:14 -04:00
|
|
|
|
|
2019-03-10 22:38:26 -04:00
|
|
|
|
$ wget --quiet -O speakers.html \
|
|
|
|
|
https://libreplanet.org/2019/speakers/ \
|
|
|
|
|
&& grep --quiet 'free software' speakers.html \
|
|
|
|
|
|| echo https://libreplanet.org/2019/speakers/ \
|
|
|
|
|
>> results.txt
|
|
|
|
|
#+END_SRC
|
2019-03-12 00:35:14 -04:00
|
|
|
|
\subskip
|
2019-03-10 22:38:26 -04:00
|
|
|
|
#+BEAMER: \end{onlyenv}
|
|
|
|
|
|
|
|
|
|
#+BEAMER: \begin{onlyenv}<1>
|
|
|
|
|
#+BEGIN_SRC sh
|
2019-03-10 23:18:05 -04:00
|
|
|
|
|
2019-03-12 00:35:14 -04:00
|
|
|
|
|
2019-03-10 22:38:26 -04:00
|
|
|
|
$ URL=https://libreplanet.org/2019/speakers/
|
|
|
|
|
$ wget --quiet -O speakers.html \
|
|
|
|
|
"$URL" \
|
|
|
|
|
&& grep --quiet 'free software' speakers.html \
|
|
|
|
|
|| echo "$URL" \
|
|
|
|
|
>> results.txt
|
|
|
|
|
#+END_SRC
|
|
|
|
|
#+BEAMER: \end{onlyenv}
|
|
|
|
|
|
|
|
|
|
#+BEAMER: \begin{onlyenv}<2>
|
|
|
|
|
#+BEGIN_SRC sh
|
2019-03-10 23:18:05 -04:00
|
|
|
|
|
2019-03-12 00:35:14 -04:00
|
|
|
|
|
2019-03-10 22:38:26 -04:00
|
|
|
|
$ URL=https://libreplanet.org/2019/speakers/
|
|
|
|
|
$ wget -qO speakers.html \
|
|
|
|
|
"$URL" \
|
|
|
|
|
&& grep -q 'free software' speakers.html \
|
|
|
|
|
|| echo "$URL" \
|
|
|
|
|
>> results.txt
|
|
|
|
|
#+END_SRC
|
|
|
|
|
#+BEAMER: \end{onlyenv}
|
|
|
|
|
|
|
|
|
|
#+BEAMER: \begin{onlyenv}<3>
|
|
|
|
|
#+BEGIN_SRC sh
|
2019-03-10 23:18:05 -04:00
|
|
|
|
|
2019-03-12 00:35:14 -04:00
|
|
|
|
|
2019-03-10 22:38:26 -04:00
|
|
|
|
$ URL=https://libreplanet.org/2019/speakers/
|
|
|
|
|
$ wget -qO - \
|
|
|
|
|
"$URL" \
|
|
|
|
|
| grep -q 'free software' \
|
|
|
|
|
|| echo "$URL" \
|
|
|
|
|
>> results.txt
|
|
|
|
|
#+END_SRC
|
|
|
|
|
#+BEAMER: \end{onlyenv}
|
|
|
|
|
|
2019-03-10 23:18:05 -04:00
|
|
|
|
#+BEAMER: \begin{onlyenv}<4>
|
2019-03-10 22:38:26 -04:00
|
|
|
|
#+BEGIN_SRC sh
|
2019-03-10 23:18:05 -04:00
|
|
|
|
|
2019-03-12 00:35:14 -04:00
|
|
|
|
|
2019-03-10 22:38:26 -04:00
|
|
|
|
$ URL=https://libreplanet.org/2019/speakers/
|
|
|
|
|
$ wget -qO - "$URL" \
|
|
|
|
|
| grep -q 'free software' || echo "$URL" >> results.txt
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2019-03-10 23:18:05 -04:00
|
|
|
|
#+END_SRC
|
|
|
|
|
#+BEAMER: \end{onlyenv}
|
|
|
|
|
|
|
|
|
|
#+BEAMER: \begin{onlyenv}<5->
|
|
|
|
|
#+BEGIN_SRC sh
|
|
|
|
|
$ alias fetch-url='wget -qO-'
|
2019-03-12 00:35:14 -04:00
|
|
|
|
|
|
|
|
|
$ URL=https://libreplanet.org/2019/speakers/
|
|
|
|
|
$ fetch-url "$URL" \
|
2019-03-10 23:18:05 -04:00
|
|
|
|
| grep -q 'free software' || echo "$URL" >> results.txt
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2019-03-10 22:38:26 -04:00
|
|
|
|
#+END_SRC
|
|
|
|
|
#+BEAMER: \end{onlyenv}
|
|
|
|
|
|
|
|
|
|
**** Notes :B_noteNH:
|
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: noteNH
|
2019-03-20 00:01:10 -04:00
|
|
|
|
:DURATION: 00:02:05
|
2019-03-10 22:38:26 -04:00
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
We can simplify it by introducing a /variable/.
|
2019-03-20 00:01:10 -04:00
|
|
|
|
First we assign the URL to a variable /named/ URL.
|
2019-03-10 22:38:26 -04:00
|
|
|
|
We then reference its value by prefixing it with a dollar sign everywhere
|
|
|
|
|
the URL previously appeared.
|
2019-03-20 00:01:10 -04:00
|
|
|
|
We put it in quotes just in case it contains special characters or
|
|
|
|
|
whitespace.
|
2019-03-10 22:38:26 -04:00
|
|
|
|
|
|
|
|
|
We can also make this command line a bit more concise by using the short
|
|
|
|
|
name for the =--quiet= flag,
|
|
|
|
|
which is =-q=.
|
|
|
|
|
Notice how in =wget= I combined them into =-qO= instead of using two
|
2019-03-20 00:01:10 -04:00
|
|
|
|
separate dashes with spaces between them,
|
|
|
|
|
but you don't have to do this.
|
2019-03-10 22:38:26 -04:00
|
|
|
|
|
|
|
|
|
Something else feels dirty.
|
|
|
|
|
We're creating this =speakers.html= file just to pass to =grep=.
|
|
|
|
|
It's not needed after the fact.
|
|
|
|
|
In fact,
|
|
|
|
|
it's just polluting our filesystem.
|
|
|
|
|
What if we didn't have to create it at all to begin with?
|
|
|
|
|
|
|
|
|
|
I'm first going to introduce the notation,
|
2019-03-20 00:01:10 -04:00
|
|
|
|
and then I'll go into why it works.
|
2019-03-10 22:38:26 -04:00
|
|
|
|
|
|
|
|
|
If we replace the output file =speakers.html= with a single dash,
|
|
|
|
|
that tells =wget= to write to standard out.
|
|
|
|
|
This is normally the default behavior of command line programs,
|
|
|
|
|
like =grep= and =echo=,
|
|
|
|
|
but =wget= is a bit different.
|
|
|
|
|
|
|
|
|
|
We then omit the =speakers.html= from =grep= entirely.
|
|
|
|
|
=grep= will read from standard in by default.
|
|
|
|
|
|
|
|
|
|
We then connect standard out of =wget= to the standard in of =grep= using a
|
2019-03-20 00:01:10 -04:00
|
|
|
|
/single/ pipe, not double;
|
2019-03-10 22:38:26 -04:00
|
|
|
|
this is called a /pipeline/.
|
|
|
|
|
|
2019-03-12 00:35:14 -04:00
|
|
|
|
Now that we've freed up some characters,
|
2019-03-10 22:38:26 -04:00
|
|
|
|
let's reformat this slightly to be a bit more readable.
|
2019-03-12 00:35:14 -04:00
|
|
|
|
And that =wget= command looks a bit cryptic.
|
2019-03-10 23:18:05 -04:00
|
|
|
|
How about we define an alias so that it looks a bit more friendly,
|
|
|
|
|
and then we can stop worrying about what it does?
|
2019-03-10 22:38:26 -04:00
|
|
|
|
|
2019-03-12 00:35:14 -04:00
|
|
|
|
Now here's the original command we started with,
|
2019-03-10 22:38:26 -04:00
|
|
|
|
and where we're at now.
|
|
|
|
|
|
|
|
|
|
This little bit of abstraction has made our intent even more clear.
|
|
|
|
|
It can now clearly be read that we're defining a URL,
|
|
|
|
|
retrieving that URL,
|
|
|
|
|
searching for a term,
|
|
|
|
|
and then appending it to a file on a non-match.
|
|
|
|
|
|
2019-03-20 00:01:10 -04:00
|
|
|
|
By the way,
|
|
|
|
|
for privacy reasons,
|
|
|
|
|
you can prefix `wget` with `torify` to run it through Tor,
|
|
|
|
|
if you have it installed.
|
|
|
|
|
|
2019-03-10 22:38:26 -04:00
|
|
|
|
But before we keep going,
|
|
|
|
|
I want to go back to a point I mentioned previously.
|
|
|
|
|
|
2019-03-25 12:53:44 -04:00
|
|
|
|
*** REHEARSED Again: Text is a Universal Interface :B_againframe:
|
2019-03-10 22:38:26 -04:00
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: againframe
|
|
|
|
|
:BEAMER_ref: *Text is a Universal Interface
|
|
|
|
|
:BEAMER_act:
|
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
**** Notes :B_noteNH:
|
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: noteNH
|
2019-03-12 00:35:14 -04:00
|
|
|
|
:DURATION: 00:00:20
|
2019-03-10 22:38:26 -04:00
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
Text is a universal interface.
|
|
|
|
|
|
|
|
|
|
Notice how we started out our journey manually inspecting text,
|
|
|
|
|
and began replacing the human part of the workflow at each step with a
|
|
|
|
|
command.
|
|
|
|
|
That's because text is something that both humans and computers can work
|
|
|
|
|
with easily.
|
|
|
|
|
|
|
|
|
|
This is a fundamental design principle in the Unix tools that I have begun
|
|
|
|
|
to present to you.
|
|
|
|
|
|
2019-03-25 12:53:44 -04:00
|
|
|
|
*** REHEARSED Pipelines :B_fullframe:
|
2019-03-10 22:38:26 -04:00
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: fullframe
|
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
#+BEAMER: \fullslidetext
|
|
|
|
|
/``Expect the output of every program to become the input to another''/
|
|
|
|
|
|
|
|
|
|
#+BEAMER: \fullsubtext\hfill
|
|
|
|
|
---Doug McIlroy (1978)
|
|
|
|
|
|
|
|
|
|
**** Notes :B_noteNH:
|
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: noteNH
|
2019-03-12 00:35:14 -04:00
|
|
|
|
:DURATION: 00:00:15
|
2019-03-10 22:38:26 -04:00
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
The invention of the Unix pipe is credited to Doug McIlroy.
|
|
|
|
|
As part of the Unix philosophy,
|
|
|
|
|
he stated:
|
|
|
|
|
``expect the output of every program to become the input to another''.
|
|
|
|
|
|
|
|
|
|
More broadly,
|
|
|
|
|
the Unix philosophy can be summarized as:
|
|
|
|
|
|
2019-03-25 12:53:44 -04:00
|
|
|
|
*** REHEARSED Summary of the Unix Philosophy :B_fullframe:
|
2019-03-10 22:38:26 -04:00
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: fullframe
|
2019-03-20 00:01:10 -04:00
|
|
|
|
:DURATION: 0:00:40
|
2019-03-10 22:38:26 -04:00
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
#+BEAMER: \begingroup\fullslidetext
|
|
|
|
|
The Unix Philosophy
|
|
|
|
|
#+BEAMER: \endgroup\subskip
|
|
|
|
|
|
|
|
|
|
#+BEGIN_QUOTE
|
|
|
|
|
This is the Unix philosophy: Write programs that do one thing and do it
|
|
|
|
|
well. Write programs to work together. Write programs to handle text
|
|
|
|
|
streams, because that is a universal interface.
|
|
|
|
|
|
|
|
|
|
#+BEAMER: \smallsubskip\hfill---Doug McIlroy
|
|
|
|
|
#+END_QUOTE
|
|
|
|
|
|
|
|
|
|
**** Notes :B_noteNH:
|
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: noteNH
|
2019-03-20 00:01:10 -04:00
|
|
|
|
:DURATION: 00:00:40
|
2019-03-10 22:38:26 -04:00
|
|
|
|
:END:
|
|
|
|
|
|
2019-03-12 00:35:14 -04:00
|
|
|
|
/<Read it>/
|
|
|
|
|
|
2019-03-20 00:01:10 -04:00
|
|
|
|
Up until this point,
|
|
|
|
|
we have changed how we communicate with the machine by moving away from a
|
2019-03-10 22:38:26 -04:00
|
|
|
|
visual interface driven primarily by movement,
|
|
|
|
|
to a textual interface that puts mind and machine on equal footing.
|
2019-03-12 00:35:14 -04:00
|
|
|
|
And now here we're talking about another profound shift in how we think.
|
2019-03-10 22:38:26 -04:00
|
|
|
|
|
|
|
|
|
We start to think of how to decompose problems into small operations that
|
|
|
|
|
exist as part of a larger pipeline.
|
2019-03-12 00:35:14 -04:00
|
|
|
|
We think of how to chain small, specialized programs together,
|
|
|
|
|
transforming text at each step to make it more suitable for the next.
|
2019-03-10 22:38:26 -04:00
|
|
|
|
|
2019-03-25 12:53:44 -04:00
|
|
|
|
** REHEARSED Program Composition [7/7]
|
2019-03-15 00:46:44 -04:00
|
|
|
|
*** Composition Topics [6/6] :noexport:
|
2019-03-14 22:57:37 -04:00
|
|
|
|
- [X] Clarify how pipelines work with existing =wget | grep=.
|
|
|
|
|
- [X] More involved pipeline with more than two programs.
|
2019-03-15 00:46:44 -04:00
|
|
|
|
- [X] Emphasize iterative development and how the shell is a REPL.
|
2019-03-14 22:57:37 -04:00
|
|
|
|
- Useful for programmers for prototyping and debugging, but also essential
|
|
|
|
|
to average users for discovery.
|
2019-03-15 00:46:44 -04:00
|
|
|
|
- Evolve by making portions of command dynamic (variables, subshells)
|
2019-03-14 22:57:37 -04:00
|
|
|
|
- [X] Now script discovering what pages contain a certain word [3/3]
|
|
|
|
|
- [X] Mention previous example of being emailed a list of URLs. Rather
|
|
|
|
|
than pasting them into a file, let's discover them using the same
|
|
|
|
|
tool: =grep=. Save email to a file.
|
|
|
|
|
- [X] =grep -o 'https\?://[^ ]\+'=, show how it gets a list of URLs.
|
|
|
|
|
- [X] Introduce =while= and =read=, showing how we can iteratively run
|
|
|
|
|
commands, maybe =... | while read url; do echo "URL: $url"; done=.
|
|
|
|
|
- Not enough talk time to walk away and get a coffee, but let's see if
|
|
|
|
|
we can maybe let it run for 10--30s while I blabber on. Depends on
|
|
|
|
|
the connection speed at MIT with all the participants.
|
|
|
|
|
- [X] Extract =url-grep= into script.
|
|
|
|
|
- [X] Demonstrate running jobs in parallel with =xargs=.
|
|
|
|
|
|
2019-03-25 12:53:44 -04:00
|
|
|
|
*** REHEARSED LP Sessions :B_fullframe:
|
2019-03-14 22:57:37 -04:00
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: fullframe
|
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
#+BEAMER: \begin{onlyenv}<+>
|
|
|
|
|
#+BEGIN_SRC sh
|
|
|
|
|
$ fetch-url https://libreplanet.org/2019/speakers/ \
|
|
|
|
|
| grep -A5 speaker-header \
|
|
|
|
|
| head -n14
|
|
|
|
|
<header class="keynote-speaker-header" id="garbee">
|
|
|
|
|
<hgroup>
|
|
|
|
|
<h2>Bdale Garbee</h2>
|
|
|
|
|
</hgroup>
|
|
|
|
|
</header>
|
|
|
|
|
<p><em>Closing keynote</em></p>
|
|
|
|
|
--
|
|
|
|
|
<header class="keynote-speaker-header" id="loubani">
|
|
|
|
|
<hgroup>
|
|
|
|
|
<h2>Tarek Loubani</h2>
|
|
|
|
|
</hgroup>
|
|
|
|
|
</header>
|
|
|
|
|
<p><em>Opening keynote (Day 1)</em></p>
|
|
|
|
|
--
|
|
|
|
|
#+END_SRC
|
|
|
|
|
#+BEAMER: \end{onlyenv}
|
|
|
|
|
|
|
|
|
|
#+BEAMER: \begin{onlyenv}<+>
|
|
|
|
|
#+BEGIN_SRC sh
|
|
|
|
|
$ fetch-url https://libreplanet.org/2019/speakers/ \
|
|
|
|
|
| grep -A5 speaker-header \
|
|
|
|
|
| grep '<em>'
|
|
|
|
|
<p><em>Closing keynote</em></p>
|
|
|
|
|
<p><em>Opening keynote (Day 1)</em></p>
|
|
|
|
|
<p><em>Opening keynote (Day 2)</em></p>
|
|
|
|
|
[...]
|
|
|
|
|
<p><em>The Tor Project: State of the Onion</em> and <em>Library Freedom Institute: A new hope</em></p>
|
|
|
|
|
<p><em>The Tor Project: State of the Onion</em></p>
|
|
|
|
|
[...]
|
|
|
|
|
<p><em>Large-scale collaboration with free software</em></p>
|
|
|
|
|
<p><em>Large-scale collaboration with free software</em></p>
|
|
|
|
|
#+END_SRC
|
|
|
|
|
#+BEAMER: \end{onlyenv}
|
|
|
|
|
|
|
|
|
|
#+BEAMER: \begin{onlyenv}<+>
|
|
|
|
|
#+BEGIN_SRC sh
|
|
|
|
|
$ fetch-url https://libreplanet.org/2019/speakers/ \
|
|
|
|
|
| grep -A5 speaker-header \
|
|
|
|
|
| grep -o '<em>[^<]\+</em>'
|
|
|
|
|
<em>Closing keynote</em>
|
|
|
|
|
<em>Opening keynote (Day 1)</em>
|
|
|
|
|
<em>Opening keynote (Day 2)</em>
|
|
|
|
|
[...]
|
|
|
|
|
<em>The Tor Project: State of the Onion</em>
|
|
|
|
|
<em>Library Freedom Institute: A new hope</em>
|
|
|
|
|
<em>The Tor Project: State of the Onion</em>
|
|
|
|
|
[...]
|
|
|
|
|
<em>Large-scale collaboration with free software</em>
|
|
|
|
|
<em>Large-scale collaboration with free software</em>
|
|
|
|
|
#+END_SRC
|
|
|
|
|
#+BEAMER: \end{onlyenv}
|
|
|
|
|
|
|
|
|
|
#+BEAMER: \begin{onlyenv}<+>
|
|
|
|
|
#+BEGIN_SRC sh
|
|
|
|
|
$ fetch-url https://libreplanet.org/2019/speakers/ \
|
|
|
|
|
| grep -A5 speaker-header \
|
|
|
|
|
| grep -o '<em>[^<]\+</em>' \
|
|
|
|
|
| sort \
|
|
|
|
|
| uniq -cd
|
|
|
|
|
2 <em>Hackerspace Rancho Electrónico</em>
|
|
|
|
|
4 <em>Large-scale collaboration with free software</em>
|
|
|
|
|
2 <em>Library Freedom Institute: A new hope</em>
|
|
|
|
|
2 <em>Right to Repair and the DMCA</em>
|
|
|
|
|
2 <em>Teaching privacy and security via free software</em>
|
|
|
|
|
2 <em>The joy of bug reporting</em>
|
|
|
|
|
5 <em>The Tor Project: State of the Onion</em>
|
|
|
|
|
#+END_SRC
|
|
|
|
|
#+BEAMER: \end{onlyenv}
|
|
|
|
|
|
|
|
|
|
#+BEAMER: \begin{onlyenv}<+>
|
|
|
|
|
#+BEGIN_SRC sh
|
|
|
|
|
$ fetch-url https://libreplanet.org/2019/speakers/ \
|
|
|
|
|
| grep -A5 speaker-header \
|
|
|
|
|
| grep -o '<em>[^<]\+</em>' \
|
|
|
|
|
| sort \
|
|
|
|
|
| uniq -cd \
|
|
|
|
|
| sort -nr \
|
|
|
|
|
| head -n5
|
|
|
|
|
5 <em>The Tor Project: State of the Onion</em>
|
|
|
|
|
4 <em>Large-scale collaboration with free software</em>
|
|
|
|
|
2 <em>The joy of bug reporting</em>
|
|
|
|
|
2 <em>Teaching privacy and security via free software</em>
|
|
|
|
|
2 <em>Right to Repair and the DMCA</em>
|
|
|
|
|
|
|
|
|
|
#+END_SRC
|
|
|
|
|
#+BEAMER: \end{onlyenv}
|
|
|
|
|
|
|
|
|
|
#+BEAMER: \begin{onlyenv}<+>
|
|
|
|
|
#+BEGIN_SRC sh
|
|
|
|
|
$ fetch-url https://libreplanet.org/2019/speakers/ \
|
|
|
|
|
| grep -A5 speaker-header \
|
|
|
|
|
| grep -o '<em>[^<]\+</em>' \
|
|
|
|
|
| sort \
|
|
|
|
|
| uniq -cd \
|
|
|
|
|
| sort -nr \
|
|
|
|
|
| head -n5 \
|
2019-03-25 12:53:44 -04:00
|
|
|
|
| sed 's#^ *\(.\+\) <em>\(.*\)</em>#\2 has \1 speakers#'
|
2019-03-14 22:57:37 -04:00
|
|
|
|
The Tor Project: State of the Onion has 5 speakers
|
|
|
|
|
Large-scale collaboration with free software has 4 speakers
|
|
|
|
|
The joy of bug reporting has 2 speakers
|
|
|
|
|
Teaching privacy and security via free software has 2 speakers
|
|
|
|
|
Right to Repair and the DMCA has 2 speakers
|
|
|
|
|
#+END_SRC
|
|
|
|
|
#+BEAMER: \end{onlyenv}
|
|
|
|
|
|
|
|
|
|
#+BEAMER: \begin{onlyenv}<+>
|
|
|
|
|
#+BEGIN_SRC sh
|
|
|
|
|
$ fetch-url https://libreplanet.org/2019/speakers/ \
|
|
|
|
|
| grep -A5 speaker-header \
|
|
|
|
|
| grep -o '<em>[^<]\+</em>' \
|
|
|
|
|
| sort \
|
|
|
|
|
| uniq -cd \
|
|
|
|
|
| sort -nr \
|
|
|
|
|
| head -n5 \
|
2019-03-20 00:01:10 -04:00
|
|
|
|
| sed 's#^ *\(.\+\) <em>\(.*\)</em>#\2 has \1 speakers#'
|
2019-03-14 22:57:37 -04:00
|
|
|
|
| espeak
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
#+END_SRC
|
|
|
|
|
#+BEAMER: \end{onlyenv}
|
|
|
|
|
|
2019-03-15 00:46:44 -04:00
|
|
|
|
**** Notes :B_noteNH:
|
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: noteNH
|
2019-03-20 00:01:10 -04:00
|
|
|
|
:DURATION: 00:02:30
|
2019-03-15 00:46:44 -04:00
|
|
|
|
:END:
|
|
|
|
|
|
2019-03-20 00:01:10 -04:00
|
|
|
|
Let's look at a more sophisticated pipeline with another practical example.
|
|
|
|
|
|
2019-03-15 23:49:15 -04:00
|
|
|
|
I noticed that some LibrePlanet sessions had multiple speakers,
|
2019-03-15 00:46:44 -04:00
|
|
|
|
and I wanted to know which ones had the /most/ speakers.
|
|
|
|
|
|
|
|
|
|
The HTML of the speakers page includes a header for each speaker.
|
|
|
|
|
Here are the first two.
|
2019-03-20 00:01:10 -04:00
|
|
|
|
=-A5= tells =grep= to also output the five lines following matching lines,
|
|
|
|
|
and =head -n14= outputs only the first fourteen lines of output.
|
2019-03-15 00:46:44 -04:00
|
|
|
|
|
2019-03-20 00:01:10 -04:00
|
|
|
|
We're only interested in the talk titles, though.
|
2019-03-15 00:46:44 -04:00
|
|
|
|
Looking at this output,
|
2019-03-20 00:01:10 -04:00
|
|
|
|
we see that the titles have an =em= tag,
|
2019-03-15 00:46:44 -04:00
|
|
|
|
so let's just go with that.
|
2019-03-15 23:49:15 -04:00
|
|
|
|
Pipe to =grep= instead of =head=.
|
2019-03-15 00:46:44 -04:00
|
|
|
|
|
2019-03-20 00:01:10 -04:00
|
|
|
|
It looks like at least one of those results has /multiple/ talks,
|
|
|
|
|
if you can see the ``and'' there.
|
2019-03-15 00:46:44 -04:00
|
|
|
|
But note that each is enclosed in its own set of =em= tags.
|
|
|
|
|
If we add =-o= to =grep=,
|
|
|
|
|
which stands for /only/,
|
|
|
|
|
then it'll only return the portion of the line that matches,
|
|
|
|
|
rather than the entire line.
|
|
|
|
|
Further,
|
|
|
|
|
if there are multiple matches on a line,
|
|
|
|
|
it'll output each match independently on its own line.
|
|
|
|
|
That's exactly what we want!
|
|
|
|
|
But we have to modify our regex a little bit to prevent it from grabbing
|
2019-03-20 00:01:10 -04:00
|
|
|
|
everything between the first and /last/ =em= tag.
|
2019-03-15 23:49:15 -04:00
|
|
|
|
Don't worry if you don't understand the regular expression;
|
|
|
|
|
they take time to learn and tend to be easier to write than they are to
|
|
|
|
|
read.
|
|
|
|
|
This one just says ``match one or more non-less-than characters between =em=
|
|
|
|
|
tags''.
|
2019-03-15 00:46:44 -04:00
|
|
|
|
|
|
|
|
|
Now assuming that the talk titles are consistent,
|
|
|
|
|
we can get a count.
|
|
|
|
|
=uniq= has the ability to count consecutive lines that are identical,
|
|
|
|
|
as well as output a count.
|
2019-03-15 23:49:15 -04:00
|
|
|
|
We also use =-d= to tell it to only output duplicate lines.
|
|
|
|
|
But =uniq= doesn't sort lines before processing,
|
2019-03-15 00:46:44 -04:00
|
|
|
|
so we first pipe it to =sort=.
|
|
|
|
|
That gives us a count of each talk!
|
|
|
|
|
|
2019-03-15 23:49:15 -04:00
|
|
|
|
But I want to know the talks with the /most/ speakers,
|
2019-03-15 00:46:44 -04:00
|
|
|
|
so let's sort it /again/,
|
2019-03-15 23:49:15 -04:00
|
|
|
|
this time numerically and in reverse order,
|
|
|
|
|
and take the top five.
|
2019-03-15 00:46:44 -04:00
|
|
|
|
|
|
|
|
|
And we have our answer!
|
|
|
|
|
|
|
|
|
|
But just for the hell of it,
|
|
|
|
|
let's go a step further.
|
|
|
|
|
Using =sed=,
|
|
|
|
|
which stands for /stream editor/,
|
|
|
|
|
we can match on portions of the input and reference those matches in a
|
|
|
|
|
replacement.
|
|
|
|
|
So we can reformat the =uniq= output into an English sentence,
|
|
|
|
|
like so.
|
2019-03-15 23:49:15 -04:00
|
|
|
|
I chose the pound characters delimit the match from the replacement.
|
|
|
|
|
The numbers in the replacement reference the parenthesized groups in the
|
|
|
|
|
match.
|
2019-03-15 00:46:44 -04:00
|
|
|
|
|
|
|
|
|
And then we're going to pipe it to the program =espeak=,
|
|
|
|
|
which is a text-to-speech synthesizer.
|
|
|
|
|
Your computer will speak the top five talks by presenter count to you.
|
|
|
|
|
Listening to computers speak is all the rage right now,
|
|
|
|
|
right?
|
|
|
|
|
|
2019-03-25 12:53:44 -04:00
|
|
|
|
*** REHEARSED Interactive, Incremental, Iterative Development :B_fullframe:
|
2019-03-14 22:57:37 -04:00
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: fullframe
|
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
#+BEAMER: \fullslidetext
|
|
|
|
|
Incremental Development
|
|
|
|
|
|
|
|
|
|
#+BEAMER: \fullsubtext
|
|
|
|
|
Interactive REPL, Iterative Decomposition
|
|
|
|
|
|
2019-03-15 00:46:44 -04:00
|
|
|
|
**** Notes :B_noteNH:
|
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: noteNH
|
2019-03-15 23:49:15 -04:00
|
|
|
|
:DURATION: 00:01:10
|
2019-03-15 00:46:44 -04:00
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
Notice how we approached that problem.
|
|
|
|
|
I presented it here just as I developed it.
|
|
|
|
|
I didn't open my web browser and inspect the HTML;
|
|
|
|
|
I just looked at the =wget= output and then started to manipulate it in
|
|
|
|
|
useful ways working toward my final goal.
|
2019-03-15 23:49:15 -04:00
|
|
|
|
This is just /one/ of the many ways to write it.
|
2019-03-15 00:46:44 -04:00
|
|
|
|
And this is part of what makes working in a shell so powerful.
|
|
|
|
|
|
|
|
|
|
In software development,
|
|
|
|
|
we call environments like this REPLs,
|
|
|
|
|
which stands for ``read-eval-print loop''.
|
|
|
|
|
The shell reads a command line,
|
|
|
|
|
evaluates it,
|
|
|
|
|
prints a result,
|
|
|
|
|
and then does that all over again.
|
|
|
|
|
As a hacker,
|
|
|
|
|
this allows me to easily inspect and iterate on my script in real time,
|
|
|
|
|
which can be a very efficient process.
|
|
|
|
|
I can quickly prototype something and then clean it up later.
|
|
|
|
|
Or maybe create a proof-of-concept in shell before writing the actual
|
|
|
|
|
implementation in another language.
|
|
|
|
|
|
|
|
|
|
But most users aren't programmers.
|
|
|
|
|
They aren't experts in these commands;
|
|
|
|
|
they have to play around and discover as they go.
|
|
|
|
|
And the shell is perfect for this discovery.
|
|
|
|
|
If something doesn't work,
|
|
|
|
|
just keep trying different things and get immediate feedback!
|
|
|
|
|
|
2019-03-15 23:49:15 -04:00
|
|
|
|
And because we're working with text as data,
|
|
|
|
|
a human can replace any part of this process!
|
|
|
|
|
|
2019-03-25 12:53:44 -04:00
|
|
|
|
*** REHEARSED Discovering URLs :B_fullframe:
|
2019-03-14 22:57:37 -04:00
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: fullframe
|
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
#+BEAMER: \begin{onlyenv}<+>
|
|
|
|
|
#+BEGIN_SRC sh
|
|
|
|
|
|
2019-03-15 23:49:15 -04:00
|
|
|
|
$ grep -o 'https\?://[^ ]\+' email-of-links.txt
|
2019-03-14 22:57:37 -04:00
|
|
|
|
https://en.wikipedia.org/wiki/Free_software
|
|
|
|
|
https://en.wikipedia.org/wiki/Open_source
|
|
|
|
|
https://en.wikipedia.org/wiki/Microsoft
|
|
|
|
|
https://opensource.org/about
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2019-03-20 00:01:10 -04:00
|
|
|
|
|
2019-03-14 22:57:37 -04:00
|
|
|
|
#+END_SRC
|
|
|
|
|
#+BEAMER: \end{onlyenv}
|
|
|
|
|
|
|
|
|
|
#+BEAMER: \begin{onlyenv}<+>
|
|
|
|
|
#+BEGIN_SRC sh
|
|
|
|
|
|
2019-03-15 23:49:15 -04:00
|
|
|
|
$ grep -o 'https\?://[^ ]\+' email-of-links.txt \
|
2019-03-14 22:57:37 -04:00
|
|
|
|
| while read URL; do
|
|
|
|
|
echo "URL is $URL"
|
|
|
|
|
done
|
|
|
|
|
URL is https://en.wikipedia.org/wiki/Free_software
|
|
|
|
|
URL is https://en.wikipedia.org/wiki/Open_source
|
|
|
|
|
URL is https://en.wikipedia.org/wiki/Microsoft
|
|
|
|
|
URL is https://opensource.org/about
|
|
|
|
|
|
2019-03-20 00:01:10 -04:00
|
|
|
|
|
2019-03-14 22:57:37 -04:00
|
|
|
|
#+END_SRC
|
|
|
|
|
#+BEAMER: \end{onlyenv}
|
|
|
|
|
|
|
|
|
|
#+BEAMER: \begin{onlyenv}<+>
|
|
|
|
|
#+BEGIN_SRC sh
|
|
|
|
|
|
2019-03-15 23:49:15 -04:00
|
|
|
|
$ grep -o 'https\?://[^ ]\+' email-of-links.txt \
|
2019-03-20 00:01:10 -04:00
|
|
|
|
| while read URL; do \
|
2019-03-14 22:57:37 -04:00
|
|
|
|
fetch-url "$URL" | grep -q 'free software' \
|
2019-03-20 00:01:10 -04:00
|
|
|
|
|| echo "$URL \
|
|
|
|
|
done \
|
|
|
|
|
> results.txt
|
2019-03-14 22:57:37 -04:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
#+END_SRC
|
|
|
|
|
#+BEAMER: \end{onlyenv}
|
|
|
|
|
|
|
|
|
|
#+BEAMER: \begin{onlyenv}<+>
|
|
|
|
|
#+BEGIN_SRC sh
|
|
|
|
|
|
2019-03-15 23:49:15 -04:00
|
|
|
|
$ grep -o 'https\?://[^ ]\+' email-of-links.txt \
|
2019-03-20 00:01:10 -04:00
|
|
|
|
| while read URL; do \
|
2019-03-14 22:57:37 -04:00
|
|
|
|
fetch-url "$URL" | grep -q 'free software' \
|
2019-03-20 00:01:10 -04:00
|
|
|
|
|| echo "$URL" \
|
|
|
|
|
done \
|
|
|
|
|
| tee results.txt
|
2019-03-14 22:57:37 -04:00
|
|
|
|
https://en.wikipedia.org/wiki/Microsoft
|
|
|
|
|
https://opensource.org/about
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
#+END_SRC
|
|
|
|
|
#+BEAMER: \end{onlyenv}
|
|
|
|
|
|
|
|
|
|
#+BEAMER: \begin{onlyenv}<+>
|
|
|
|
|
#+BEGIN_SRC sh
|
|
|
|
|
|
2019-03-15 23:49:15 -04:00
|
|
|
|
$ grep -o 'https\?://[^ ]\+' email-of-links.txt \
|
2019-03-20 00:01:10 -04:00
|
|
|
|
| while read URL; do \
|
2019-03-14 22:57:37 -04:00
|
|
|
|
fetch-url "$URL" | grep -q 'free software' \
|
2019-03-20 00:01:10 -04:00
|
|
|
|
|| echo "$URL" \
|
2019-03-14 22:57:37 -04:00
|
|
|
|
done \
|
2019-03-20 00:01:10 -04:00
|
|
|
|
| tee results.txt
|
2019-03-14 22:57:37 -04:00
|
|
|
|
| xclip -i -selection clipboard
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
#+END_SRC
|
|
|
|
|
#+BEAMER: \end{onlyenv}
|
|
|
|
|
|
|
|
|
|
#+BEAMER: \begin{onlyenv}<+>
|
|
|
|
|
#+BEGIN_SRC sh
|
|
|
|
|
|
2019-03-15 23:49:15 -04:00
|
|
|
|
$ grep -o 'https\?://[^ ]\+' email-of-links.txt \
|
2019-03-20 00:01:10 -04:00
|
|
|
|
| while read URL; do \
|
2019-03-14 22:57:37 -04:00
|
|
|
|
fetch-url "$URL" | grep -q 'free software' \
|
2019-03-20 00:01:10 -04:00
|
|
|
|
|| echo "$URL" \
|
2019-03-14 22:57:37 -04:00
|
|
|
|
done \
|
|
|
|
|
| tee >( xclip -i -selection clipboard )
|
|
|
|
|
https://en.wikipedia.org/wiki/Microsoft
|
|
|
|
|
https://opensource.org/about
|
|
|
|
|
|
2019-03-20 00:01:10 -04:00
|
|
|
|
|
2019-03-14 22:57:37 -04:00
|
|
|
|
#+END_SRC
|
|
|
|
|
#+BEAMER: \end{onlyenv}
|
|
|
|
|
|
|
|
|
|
#+BEAMER: \begin{onlyenv}<+>
|
|
|
|
|
#+BEGIN_SRC sh
|
|
|
|
|
$ xclip -o -selection clipboard \
|
|
|
|
|
| grep -o 'https\?://[^ ]\+' \
|
2019-03-20 00:01:10 -04:00
|
|
|
|
| while read URL; do \
|
2019-03-14 22:57:37 -04:00
|
|
|
|
fetch-url "$URL" | grep -q 'free software' \
|
2019-03-20 00:01:10 -04:00
|
|
|
|
|| echo "$URL" \
|
2019-03-14 22:57:37 -04:00
|
|
|
|
done \
|
|
|
|
|
| tee >( xclip -i -selection clipboard )
|
|
|
|
|
https://en.wikipedia.org/wiki/Microsoft
|
|
|
|
|
https://opensource.org/about
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
#+END_SRC
|
|
|
|
|
#+BEAMER: \end{onlyenv}
|
|
|
|
|
|
|
|
|
|
#+BEAMER: \begin{onlyenv}<+>
|
|
|
|
|
#+BEGIN_SRC sh
|
|
|
|
|
$ xclip -o -selection clipboard \
|
|
|
|
|
| grep -o 'https\?://[^ ]\+' \
|
2019-03-20 00:01:10 -04:00
|
|
|
|
| while read URL; do \
|
2019-03-14 22:57:37 -04:00
|
|
|
|
fetch-url "$URL" | grep -q 'free software' \
|
2019-03-20 00:01:10 -04:00
|
|
|
|
|| echo "$URL" \
|
|
|
|
|
done \
|
|
|
|
|
| tee results.txt
|
2019-03-14 22:57:37 -04:00
|
|
|
|
https://en.wikipedia.org/wiki/Microsoft
|
|
|
|
|
https://opensource.org/about
|
|
|
|
|
|
|
|
|
|
$ xclip -i -selection clipboard < results.txt
|
|
|
|
|
#+END_SRC
|
|
|
|
|
#+BEAMER: \end{onlyenv}
|
|
|
|
|
|
2019-03-15 00:46:44 -04:00
|
|
|
|
**** Notes :B_noteNH:
|
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: noteNH
|
2019-03-20 00:01:10 -04:00
|
|
|
|
:DURATION: 00:02:15
|
2019-03-15 00:46:44 -04:00
|
|
|
|
:END:
|
|
|
|
|
|
2019-03-15 23:49:15 -04:00
|
|
|
|
Okay, back to searching webpages.
|
|
|
|
|
Now that we have a means of creating the list of results,
|
|
|
|
|
how do we feed the URLs into our pipeline?
|
2019-03-15 00:46:44 -04:00
|
|
|
|
Why not pull them right out of the email with =grep=?
|
|
|
|
|
|
2019-03-15 23:49:15 -04:00
|
|
|
|
Let's say you saved the email in =email-of-links.txt=.
|
2019-03-22 00:21:41 -04:00
|
|
|
|
This simple regex should grab most URLs,
|
2019-03-15 23:49:15 -04:00
|
|
|
|
but it's far from perfect.
|
|
|
|
|
For example,
|
|
|
|
|
it'd grab punctuation at the end of a sentence.
|
|
|
|
|
But we're assuming a list of URLs.
|
2019-03-20 00:01:10 -04:00
|
|
|
|
Here's some example output with a few.
|
2019-03-15 00:46:44 -04:00
|
|
|
|
|
|
|
|
|
For each of these,
|
|
|
|
|
we need to run our pipeline.
|
|
|
|
|
It's time to introduce =while= and =read=.
|
|
|
|
|
=while= will continue to execute its body in a loop until its command fails.
|
|
|
|
|
=read= will read line-by-line into one or more variables,
|
|
|
|
|
and will fail when there are no more lines to read.
|
|
|
|
|
|
2019-03-15 23:49:15 -04:00
|
|
|
|
So if we insert our =fetch-url= pipeline into the body,
|
2019-03-15 00:46:44 -04:00
|
|
|
|
we get this.
|
|
|
|
|
For convenience,
|
|
|
|
|
let's use =tee=,
|
|
|
|
|
which is named for a pipe tee;
|
|
|
|
|
it'll send output through the pipeline while also writing the same
|
|
|
|
|
output to a given file.
|
2019-03-15 23:49:15 -04:00
|
|
|
|
So now we can both observe the results /and/ have them written to a file!
|
2019-03-15 00:46:44 -04:00
|
|
|
|
|
|
|
|
|
But we were just going to reply to an email with those results.
|
|
|
|
|
Let's assume we're still using a GUI email client.
|
2019-03-15 23:49:15 -04:00
|
|
|
|
Wouldn't it be convenient if those results were already on the clipboard for
|
|
|
|
|
us so we can just paste them into the message?
|
2019-03-15 00:46:44 -04:00
|
|
|
|
We can accomplish that by piping to =xclip= as shown here.
|
|
|
|
|
|
|
|
|
|
Ah, crap, but now we can't see the output again.
|
2019-03-20 00:01:10 -04:00
|
|
|
|
Do we really need the results file anymore?
|
|
|
|
|
Rather than outputting to a file on disk with =tee=,
|
2019-03-15 00:46:44 -04:00
|
|
|
|
we're going to use a special notation that tells bash to invoke a command
|
|
|
|
|
in a subshell and replace that portion of the command line with a path
|
|
|
|
|
to a virtual file representing the standard input of that subshell.
|
2019-03-20 00:01:10 -04:00
|
|
|
|
This is a bash-specific feature.
|
2019-03-15 00:46:44 -04:00
|
|
|
|
Now we can see the output again!
|
|
|
|
|
|
|
|
|
|
Well,
|
|
|
|
|
if we're /writing/ to the clipboard,
|
|
|
|
|
why don't we just /read/ from it too?
|
|
|
|
|
Instead of saving our mail to a file,
|
|
|
|
|
we can just copy the relevant portion and have that piped directly to
|
|
|
|
|
=grep=!
|
2019-03-15 23:49:15 -04:00
|
|
|
|
If you have a list of URLs and you just copy that portion,
|
|
|
|
|
then you can just get rid of =grep= entirely.
|
2019-03-15 00:46:44 -04:00
|
|
|
|
|
2019-03-20 00:01:10 -04:00
|
|
|
|
What if you wrote to =results.txt= and later decided that you want to copy
|
|
|
|
|
the results to the clipboard?
|
|
|
|
|
We can do that too by reading =results.txt= in place of standard input to
|
2019-03-15 00:46:44 -04:00
|
|
|
|
=xclip=,
|
|
|
|
|
as shown here.
|
|
|
|
|
|
|
|
|
|
Phew!
|
|
|
|
|
|
2019-03-25 12:53:44 -04:00
|
|
|
|
*** REHEARSED Go Grab a Coffee :B_fullframe:
|
2019-03-14 22:57:37 -04:00
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: fullframe
|
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
#+BEAMER: \fullslidetext
|
|
|
|
|
Go Grab a Coffee
|
|
|
|
|
|
|
|
|
|
**** Notes :B_noteNH:
|
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: noteNH
|
2019-03-15 23:49:15 -04:00
|
|
|
|
:DURATION: 00:00:15
|
2019-03-14 22:57:37 -04:00
|
|
|
|
:END:
|
|
|
|
|
|
2019-03-15 00:46:44 -04:00
|
|
|
|
Remember when I said I could go grab a coffee and play with the kids while
|
|
|
|
|
the script did its thing?
|
|
|
|
|
Well now's that time.
|
2019-03-14 22:57:37 -04:00
|
|
|
|
|
|
|
|
|
But grabbing a coffee means that this system is a bottleneck.
|
2019-03-15 00:46:44 -04:00
|
|
|
|
The Internet is fast nowadays;
|
|
|
|
|
ideally, we wouldn't have to wait long.
|
2019-03-14 22:57:37 -04:00
|
|
|
|
Can we do better?
|
|
|
|
|
|
2019-03-25 12:53:44 -04:00
|
|
|
|
*** REHEARSED Async Processes :B_fullframe:
|
2019-03-14 22:57:37 -04:00
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: fullframe
|
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
#+BEGIN_SRC sh
|
|
|
|
|
$ sleep 1 && echo done & echo start
|
|
|
|
|
start
|
|
|
|
|
done
|
|
|
|
|
#+END_SRC
|
|
|
|
|
|
|
|
|
|
#+BEAMER: \begin{uncoverenv}<2>\subskip
|
|
|
|
|
/(Don't do this for large numbers of URLs!)/
|
|
|
|
|
|
|
|
|
|
#+BEGIN_SRC sh
|
|
|
|
|
$ while read URL; do
|
|
|
|
|
fetch-url "$URL" | grep -q 'free software' \
|
2019-03-20 00:01:10 -04:00
|
|
|
|
|| echo "$URL" &
|
|
|
|
|
done | tee results.txt
|
2019-03-14 22:57:37 -04:00
|
|
|
|
#+END_SRC
|
|
|
|
|
#+BEAMER: \end{uncoverenv}
|
|
|
|
|
|
2019-03-15 00:46:44 -04:00
|
|
|
|
**** Notes :B_noteNH:
|
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: noteNH
|
2019-03-20 00:01:10 -04:00
|
|
|
|
:DURATION: 00:00:45
|
2019-03-15 00:46:44 -04:00
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
What if we could query multiple URLs in parallel?
|
|
|
|
|
|
|
|
|
|
Shells have built-in support for backgrounding tasks so that they can run
|
|
|
|
|
while you do other things;
|
|
|
|
|
all you have to do is place a single ampersand at the end of a command.
|
|
|
|
|
So in this example,
|
|
|
|
|
we sleep for one second and then echo ``done''.
|
|
|
|
|
But that sleep and subsequent echo is put into the background,
|
2019-03-15 23:49:15 -04:00
|
|
|
|
and the shell proceeds to execute =echo start= while =sleep= is running in
|
|
|
|
|
the background.
|
2019-03-15 00:46:44 -04:00
|
|
|
|
One second later,
|
|
|
|
|
it outputs ``done''.
|
|
|
|
|
|
|
|
|
|
So here's the loop we were just writing.
|
|
|
|
|
If we add an ampersand at the end of that pipeline,
|
|
|
|
|
it'll run in the background and immediately proceed to the next URL,
|
|
|
|
|
executing the loop again.
|
|
|
|
|
|
|
|
|
|
But there's a problem with this approach.
|
|
|
|
|
Sure,
|
|
|
|
|
it's fine if we only have a few URLs.
|
|
|
|
|
But what if we have 1000?
|
2019-03-15 23:49:15 -04:00
|
|
|
|
That isn't efficient,
|
2019-03-20 00:01:10 -04:00
|
|
|
|
and it's a bit rude to server administrators.
|
2019-03-15 00:46:44 -04:00
|
|
|
|
|
2019-03-25 12:53:44 -04:00
|
|
|
|
*** REHEARSED Executable Shell Script and Concurrency :B_fullframe:
|
2019-03-14 22:57:37 -04:00
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: fullframe
|
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
**** =url-grep= :B_block:
|
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: block
|
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
#+BEGIN_SRC sh
|
|
|
|
|
#!/bin/bash
|
|
|
|
|
|
2019-03-20 00:01:10 -04:00
|
|
|
|
search=$1
|
|
|
|
|
url=$2
|
2019-03-14 22:57:37 -04:00
|
|
|
|
|
|
|
|
|
wget -qO- "$url" \
|
|
|
|
|
| grep -q "$search" || echo "$url"
|
|
|
|
|
#+END_SRC
|
|
|
|
|
|
|
|
|
|
**** Execute :B_ignoreheading:
|
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: ignoreheading
|
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
#+BEGIN_SRC sh
|
|
|
|
|
$ chmod +x url-grep
|
|
|
|
|
#+END_SRC
|
|
|
|
|
|
2019-03-25 12:53:44 -04:00
|
|
|
|
#+BEAMER: \begin{onlyenv}<1>
|
2019-03-14 22:57:37 -04:00
|
|
|
|
#+BEGIN_SRC sh
|
|
|
|
|
$ while read URL; do
|
|
|
|
|
./url-grep 'free software' "$URL" >> results.txt
|
|
|
|
|
done
|
|
|
|
|
#+END_SRC
|
|
|
|
|
#+BEAMER: \end{onlyenv}
|
|
|
|
|
|
2019-03-25 12:53:44 -04:00
|
|
|
|
#+BEAMER: \begin{onlyenv}<2>
|
2019-03-14 22:57:37 -04:00
|
|
|
|
#+BEGIN_SRC sh
|
|
|
|
|
$ xargs -n1 ./url-grep 'free software' > results.txt
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
#+END_SRC
|
|
|
|
|
#+BEAMER: \end{onlyenv}
|
|
|
|
|
|
2019-03-25 12:53:44 -04:00
|
|
|
|
#+BEAMER: \begin{onlyenv}<3>
|
2019-03-14 22:57:37 -04:00
|
|
|
|
#+BEGIN_SRC sh
|
|
|
|
|
$ xargs -n1 -P5 ./url-grep 'free software' > results.txt
|
|
|
|
|
# ^ 5 concurrent processes
|
|
|
|
|
|
|
|
|
|
#+END_SRC
|
|
|
|
|
#+BEAMER: \end{onlyenv}
|
|
|
|
|
|
2019-03-15 00:46:44 -04:00
|
|
|
|
**** Notes :B_noteNH:
|
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: noteNH
|
2019-03-20 00:01:10 -04:00
|
|
|
|
:DURATION: 00:01:30
|
2019-03-15 00:46:44 -04:00
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
Before we continue,
|
|
|
|
|
we're going to have to write our pipeline in a way that other programs can
|
|
|
|
|
run it.
|
|
|
|
|
Up to this point,
|
2019-03-15 23:49:15 -04:00
|
|
|
|
the program has just been embedded within an interactive shell session.
|
|
|
|
|
One of the nice things about shell is that you can take what you entered
|
|
|
|
|
onto the command line and paste it directly into a file and,
|
2019-03-15 00:46:44 -04:00
|
|
|
|
with some minor exceptions,
|
|
|
|
|
it'll work all the same.
|
|
|
|
|
|
|
|
|
|
Let's take our pipeline and name it =url-grep=.
|
2019-03-15 23:49:15 -04:00
|
|
|
|
Aliases only work in interactive sessions by default,
|
2019-03-15 00:46:44 -04:00
|
|
|
|
so we're going to just type =wget= directly here.
|
|
|
|
|
Alternatively,
|
|
|
|
|
you can define a function.
|
|
|
|
|
We use the positional parameters =1= and =2= here to represent the
|
2019-03-20 00:01:10 -04:00
|
|
|
|
respective arguments to the =url-grep= command,
|
|
|
|
|
and assign them to new variables for clarity.
|
2019-03-15 00:46:44 -04:00
|
|
|
|
|
2019-03-15 23:49:15 -04:00
|
|
|
|
The comment at the top of the file is called a ``shebang''.
|
|
|
|
|
This is used by the kernel so that it knows what interpreter to use to run
|
|
|
|
|
our program.
|
2019-03-15 00:46:44 -04:00
|
|
|
|
|
|
|
|
|
To make it executable,
|
2019-03-15 23:49:15 -04:00
|
|
|
|
we use =chmod= to set the executable bits on the file.
|
2019-03-15 00:46:44 -04:00
|
|
|
|
We can then invoke it as if it were an executable.
|
2019-03-20 00:01:10 -04:00
|
|
|
|
You actually don't /have/ to do this;
|
|
|
|
|
you can call =bash url-grep= instead of making it executable,
|
|
|
|
|
and then you don't need the shebang either.
|
2019-03-15 00:46:44 -04:00
|
|
|
|
|
|
|
|
|
Now we replace the =while= loop with =xargs=.
|
|
|
|
|
It takes values from standard in and appends them as arguments to the
|
2019-03-20 00:01:10 -04:00
|
|
|
|
provided command line.
|
2019-03-15 00:46:44 -04:00
|
|
|
|
We specify =-n1= to say that only one argument should be read from stdin
|
|
|
|
|
for any invocation of the command;
|
|
|
|
|
that makes it run a new command for every line of input.
|
|
|
|
|
Otherwise it'd just append N URLs as N arguments.
|
|
|
|
|
|
|
|
|
|
And now we can simply use =-P= to tell it how many processes to use at once.
|
|
|
|
|
Here we specify =5=,
|
|
|
|
|
meaning =xargs= will run five processes at a time.
|
|
|
|
|
You can change that to whatever number makes sense for you.
|
|
|
|
|
|
2019-03-25 12:53:44 -04:00
|
|
|
|
*** REHEARSED Execution Time :B_frame:
|
2019-03-14 22:57:37 -04:00
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: frame
|
|
|
|
|
:END:
|
|
|
|
|
|
2019-03-15 00:46:44 -04:00
|
|
|
|
#+BEGIN_SRC sh
|
|
|
|
|
$ wc -l url-list
|
|
|
|
|
1000
|
|
|
|
|
|
2019-03-20 00:01:10 -04:00
|
|
|
|
$ time xargs -n1 -P10 ./url-grep 'free software' < url-list
|
2019-03-15 00:46:44 -04:00
|
|
|
|
real 0m17.548s
|
|
|
|
|
user 0m8.283s
|
|
|
|
|
sys 0m4.877s
|
|
|
|
|
#+END_SRC
|
|
|
|
|
|
|
|
|
|
**** Notes :B_noteNH:
|
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: noteNH
|
2019-03-22 00:21:41 -04:00
|
|
|
|
:DURATION: 00:00:50
|
2019-03-15 00:46:44 -04:00
|
|
|
|
:END:
|
|
|
|
|
|
2019-03-20 00:01:10 -04:00
|
|
|
|
So how long does it take to run?
|
2019-03-15 00:46:44 -04:00
|
|
|
|
|
2019-03-15 23:49:15 -04:00
|
|
|
|
I took a few URLs and just repeated them in a file so that I had 1000 of
|
|
|
|
|
them.
|
2019-03-15 00:46:44 -04:00
|
|
|
|
Running the =xargs= command,
|
|
|
|
|
it finishes in under 18 seconds on my system at home.
|
2019-03-15 23:49:15 -04:00
|
|
|
|
Obviously YMMV,
|
|
|
|
|
and certain sites may be slower to respond than others.
|
2019-03-15 00:46:44 -04:00
|
|
|
|
|
2019-03-20 00:01:10 -04:00
|
|
|
|
If you were to write this from scratch knowing what you know now,
|
|
|
|
|
then in just a few minutes,
|
|
|
|
|
the task will have been automated away /and/ completed,
|
2019-03-15 00:46:44 -04:00
|
|
|
|
all by gluing together existing programs.
|
2019-03-20 00:01:10 -04:00
|
|
|
|
|
2019-03-15 00:46:44 -04:00
|
|
|
|
You don't need to be a programmer to know how to do this;
|
|
|
|
|
you just need to be familiar with the tools and know what's possible,
|
|
|
|
|
which comes with a little bit of practice.
|
2019-03-22 00:21:41 -04:00
|
|
|
|
It can certainly be daunting at first.
|
|
|
|
|
But I hope that by walking you through these examples and showing you how to
|
|
|
|
|
construct them step-by-step,
|
|
|
|
|
it helps to demystify them and show that there is no wizardry involved.
|
2019-03-15 00:46:44 -04:00
|
|
|
|
|
2019-03-15 23:49:15 -04:00
|
|
|
|
We've come a long way from using the web browser and a mouse.
|
2019-03-15 00:46:44 -04:00
|
|
|
|
|
2019-03-20 00:01:10 -04:00
|
|
|
|
Now that we've finally completed our research task,
|
|
|
|
|
let's look at a few more examples.
|
2019-03-10 22:38:26 -04:00
|
|
|
|
|
2019-03-25 12:53:44 -04:00
|
|
|
|
** REHEARSED More Examples [8/8]
|
2019-03-16 23:45:57 -04:00
|
|
|
|
*** More Example Topics [7/7] :noexport:
|
|
|
|
|
- [X] Screenshots with =import=.
|
|
|
|
|
- Including copying to clipboard.
|
|
|
|
|
- [X] Using =import= and =tesseract= for screenshot OCR.
|
|
|
|
|
- See if we can demo on the LP laptop.
|
|
|
|
|
- It works very well with the [[https://static.fsf.org/nosvn/libreplanet/2019/assets/keynote_banner.png][LP2019 keynote speakers image]].
|
|
|
|
|
- [X] Checking the weather
|
|
|
|
|
- [X] Resizing images
|
|
|
|
|
- [X] Password management [3/3]
|
|
|
|
|
- [X] =pwgen=.
|
|
|
|
|
- [X] Diceware/EFF word list with POSIX tools.
|
|
|
|
|
- [X] GPG-encrypted password list over SSH with smart card.
|
|
|
|
|
- [X] Full circle: automating with =xdotool=.
|
|
|
|
|
- Use =tesserac= to detect matches from =Ctrl+F=.
|
|
|
|
|
- [X] Finding documentation with =info= and =man=.
|
|
|
|
|
|
|
|
|
|
|
2019-03-25 12:53:44 -04:00
|
|
|
|
*** REHEARSED Resize Images :B_frame:
|
2019-03-16 23:45:57 -04:00
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: frame
|
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
#+BEGIN_SRC
|
|
|
|
|
$ for img in *.png; do
|
|
|
|
|
convert "$img" -resize 50% "sm-$img"
|
|
|
|
|
done
|
|
|
|
|
|
|
|
|
|
# nested directories
|
|
|
|
|
$ find . -name '*.png' -exec convert {} -resize 50% sm-{} \;
|
|
|
|
|
#+END_SRC
|
|
|
|
|
|
2019-03-17 12:14:22 -04:00
|
|
|
|
**** Notes :B_noteNH:
|
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: noteNH
|
2019-03-20 00:01:10 -04:00
|
|
|
|
:DURATION: 00:00:30
|
2019-03-17 12:14:22 -04:00
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
This example will be useful to people who have a lot of images that the want
|
|
|
|
|
to perform an operation on.
|
|
|
|
|
ImageMagick has a =convert= tool which can do a huge variety of image
|
|
|
|
|
manipulations that you would expect to have to use, say, GIMP for.
|
|
|
|
|
In this case,
|
|
|
|
|
we're just using one of the simples ones to reduce the image size by 50%.
|
|
|
|
|
|
|
|
|
|
The first example uses /globbing/ to find all PNG images in the current
|
|
|
|
|
directory.
|
|
|
|
|
The second example uses the =find= command and searches all child
|
|
|
|
|
directories as well.
|
|
|
|
|
Both examples produce a new set of images prefixed with =sm-=.
|
|
|
|
|
|
2019-03-25 12:53:44 -04:00
|
|
|
|
*** REHEARSED Password Generation :B_frame:
|
2019-03-16 23:45:57 -04:00
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: frame
|
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
#+BEGIN_SRC
|
|
|
|
|
# generate a random 32-character password
|
2019-03-17 21:21:59 -04:00
|
|
|
|
$ tr -cd '[:graph:]' < /dev/urandom | head -c32
|
|
|
|
|
`TB~cmJQ1%S8&tJ,%FoD54}"Fm4}\Iwi
|
2019-03-16 23:45:57 -04:00
|
|
|
|
|
|
|
|
|
# generate passphrase from EFF's large dice wordlist
|
|
|
|
|
# (https://www.eff.org/dice)
|
|
|
|
|
$ cut -f2 eff_large_wordlist.txt \
|
|
|
|
|
| sort -R --random-source=/dev/urandom \
|
|
|
|
|
| head -n6 \
|
|
|
|
|
| tr '\n' ' '
|
|
|
|
|
oppressor roman jigsaw unhappy thinning grievance
|
|
|
|
|
#+END_SRC
|
|
|
|
|
|
2019-03-17 12:14:22 -04:00
|
|
|
|
**** Notes :B_noteNH:
|
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: noteNH
|
2019-03-17 13:07:45 -04:00
|
|
|
|
:DURATION: 00:00:45
|
2019-03-17 12:14:22 -04:00
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
How about password generation?
|
2019-03-20 00:01:10 -04:00
|
|
|
|
=/dev/urandom= is a stream of random binary data.
|
2019-03-17 21:21:59 -04:00
|
|
|
|
We can use =tr= to delete everything that is not a printable character by
|
|
|
|
|
taking the complement of =graph=.
|
2019-03-17 12:14:22 -04:00
|
|
|
|
That type of password is useful if you have a password manager,
|
|
|
|
|
but it's not useful if you need to memorize it.
|
|
|
|
|
|
|
|
|
|
When memorization is needed,
|
|
|
|
|
a /passphrase/ may be a better option.
|
|
|
|
|
A common way to generate those is to use a large list of memorable words and
|
|
|
|
|
choose them /at random/.
|
|
|
|
|
Diceware is one such system,
|
|
|
|
|
and the EFF has its own word list.
|
|
|
|
|
But we don't need physical dice when we can just use =sort= to randomly
|
|
|
|
|
permute the word list.
|
|
|
|
|
The EFF recommends taking at least six words,
|
|
|
|
|
which is what I did here.
|
|
|
|
|
This one is particularly memorable and morbid-sounding.
|
|
|
|
|
I'm a little upset that I put it in the slide instead of using it!
|
|
|
|
|
|
2019-03-25 12:53:44 -04:00
|
|
|
|
*** REHEARSED Password Manager :B_frame:
|
2019-03-16 23:45:57 -04:00
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: frame
|
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
#+BEGIN_SRC sh
|
|
|
|
|
$ gpg --decrypt password-db.gpg | head -n3
|
|
|
|
|
https://foo.com
|
|
|
|
|
user mikegerwitz
|
|
|
|
|
pass !({:pT6DcJG.cr&OAco_EC63r_*xg|uD
|
|
|
|
|
#+END_SRC
|
|
|
|
|
|
|
|
|
|
#+BEAMER: \begin{uncoverenv}<2>
|
|
|
|
|
#+BEGIN_SRC sh
|
|
|
|
|
$ gpg --decrypt password-db.gpg \
|
|
|
|
|
| grep -A2 ^https://foo.com \
|
|
|
|
|
| tail -n2 \
|
|
|
|
|
| while read key value; do
|
|
|
|
|
echo "paste $key..."
|
|
|
|
|
printf %s "$value" \
|
|
|
|
|
| xclip -o -selection clipboard -l 1 -quiet
|
|
|
|
|
done
|
|
|
|
|
#+END_SRC
|
|
|
|
|
#+BEAMER: \end{uncoverenv}
|
|
|
|
|
|
2019-03-17 12:14:22 -04:00
|
|
|
|
**** Notes :B_noteNH:
|
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: noteNH
|
2019-03-20 00:01:10 -04:00
|
|
|
|
:DURATION: 00:01:40
|
2019-03-17 12:14:22 -04:00
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
Speaking of password managers.
|
|
|
|
|
We can create a pretty decent rather quickly.
|
|
|
|
|
Let's say we have a password list encrypted using GnuPG,
|
|
|
|
|
which can be decrypted like so.
|
|
|
|
|
Each account has a URL,
|
|
|
|
|
username,
|
|
|
|
|
and password.
|
|
|
|
|
|
|
|
|
|
We pipe the output to =grep= to find the one we're looking for.
|
|
|
|
|
The caret anchors the match to the beginning of the line.
|
|
|
|
|
We then use =tail= to keep only the last two lines,
|
|
|
|
|
discarding the URL.
|
|
|
|
|
|
2019-03-20 00:01:10 -04:00
|
|
|
|
Here we see =read= with more than one variable.
|
2019-03-17 12:14:22 -04:00
|
|
|
|
For each line,
|
2019-03-20 00:01:10 -04:00
|
|
|
|
we read the first word into =key= and the rest of the line into =value=.
|
2019-03-17 12:14:22 -04:00
|
|
|
|
This is where it gets a bit more interesting.
|
|
|
|
|
We don't want to actually output our password anywhere where others could
|
|
|
|
|
see it or where it may be vulnerable to a side channel like Van Eck
|
|
|
|
|
phreaking.
|
|
|
|
|
Instead,
|
|
|
|
|
we're going to copy the value to the clipboard.
|
2019-03-20 00:01:10 -04:00
|
|
|
|
Here we use =printf= instead of =echo= because of some technical details I
|
|
|
|
|
don't have time to get into here,
|
|
|
|
|
but it has to do with =echo= option parsing.
|
|
|
|
|
|
|
|
|
|
/(If you're reading this: imagine what happens if the password is ``-n
|
|
|
|
|
foo'', for example.)/
|
2019-03-17 12:14:22 -04:00
|
|
|
|
|
2019-03-20 00:01:10 -04:00
|
|
|
|
But the clipboard has its own risks too.
|
2019-03-17 12:14:22 -04:00
|
|
|
|
What if there's a malicious program monitoring the clipboard for passwords?
|
|
|
|
|
I used =-l 1= with =xclip=.
|
|
|
|
|
=-l= stands for ``loop'',
|
|
|
|
|
which is the number of times to serve paste requests.
|
|
|
|
|
Normally =xclip= goes into the background and acts as a server.
|
|
|
|
|
=-quiet= keeps =xclip= in the foreground.
|
|
|
|
|
So if we see that the script moves on to request pasting the next value
|
|
|
|
|
before we've actually pasted it,
|
|
|
|
|
then we'll be made aware that something is wrong and perhaps we should
|
|
|
|
|
change our account information.
|
|
|
|
|
|
2019-03-20 00:01:10 -04:00
|
|
|
|
Another nice consequence is that the first value you paste will be the
|
|
|
|
|
username,
|
|
|
|
|
and the second value will be the password.
|
2019-03-17 12:14:22 -04:00
|
|
|
|
And then the script will exit.
|
|
|
|
|
Very convenient!
|
|
|
|
|
|
|
|
|
|
Since the decrypted data exist only as part of the pipeline,
|
|
|
|
|
the decrypted passwords are only kept plaintext in memory for the duration
|
|
|
|
|
of the script.
|
|
|
|
|
|
2019-03-25 12:53:44 -04:00
|
|
|
|
*** REHEARSED Remote Password Manager With 2FA :B_frame:
|
2019-03-16 23:45:57 -04:00
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: frame
|
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
- Add =extra-socket= to =.gnupg/gpg-agent.conf=
|
|
|
|
|
- Add =RemoteForward= in =.ssh/config= for host
|
|
|
|
|
- Save script on previous slide as =get-passwd=
|
|
|
|
|
|
|
|
|
|
#+BEAMER: \subskip
|
|
|
|
|
|
|
|
|
|
#+BEGIN_SRC sh
|
|
|
|
|
$ ssh -Y mikegerwitz-pc get-passwd https://foo.com
|
|
|
|
|
#+END_SRC
|
|
|
|
|
|
2019-03-17 12:14:22 -04:00
|
|
|
|
**** Notes :B_noteNH:
|
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: noteNH
|
2019-03-17 13:07:45 -04:00
|
|
|
|
:DURATION: 00:01:30
|
2019-03-17 12:14:22 -04:00
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
But what if you need passwords on multiple devices?
|
|
|
|
|
Well,
|
|
|
|
|
then you have to worry about how to keep them in sync.
|
|
|
|
|
It also means that,
|
|
|
|
|
if you are travelling and asked to decrypt files on your laptop,
|
|
|
|
|
you're put in a tough spot.
|
|
|
|
|
|
|
|
|
|
Instead,
|
|
|
|
|
let's make that command line from the previous slide a script called
|
|
|
|
|
=get-passwd=,
|
|
|
|
|
and have it take the URL as an argument.
|
|
|
|
|
We can then access it remotely over SSH.
|
|
|
|
|
But to have clipboard access,
|
|
|
|
|
we need to forward our X11 session,
|
|
|
|
|
which is what =-Y= does.
|
|
|
|
|
|
2019-03-20 00:01:10 -04:00
|
|
|
|
But that's still not good enough.
|
2019-03-17 12:14:22 -04:00
|
|
|
|
What if my master password is compromised---the
|
|
|
|
|
password for my database?
|
|
|
|
|
I'd rather have two-factor authentication.
|
2019-03-20 00:01:10 -04:00
|
|
|
|
I asymmetrically encrypt my password database using a private key that's
|
2019-03-17 12:14:22 -04:00
|
|
|
|
stored on my Nitrokey,
|
|
|
|
|
which is a smart card---keys
|
|
|
|
|
cannot be extracted from it,
|
|
|
|
|
unless there's a vulnerabilty of course.
|
|
|
|
|
To unlock the card,
|
|
|
|
|
I must enter a PIN.
|
2019-03-20 00:01:10 -04:00
|
|
|
|
So that's both something I know and something I have.
|
2019-03-17 12:14:22 -04:00
|
|
|
|
If you enter the PIN incorrectly three times,
|
|
|
|
|
the PIN needs to be reset with an administrative password.
|
|
|
|
|
If you get that wrong three times,
|
|
|
|
|
the device bricks itself.
|
|
|
|
|
And this all works over SSH too.
|
|
|
|
|
|
2019-03-20 00:01:10 -04:00
|
|
|
|
Asymmetic encryption is also nice for shared passwords.
|
|
|
|
|
I share a small password database with my wife,
|
|
|
|
|
for example.
|
|
|
|
|
|
2019-03-17 12:14:22 -04:00
|
|
|
|
Encrypting long-term secrets with asymmetric keys isn't a great idea,
|
|
|
|
|
so there's a tradeoff.
|
|
|
|
|
I choose to accept it,
|
|
|
|
|
because passwords aren't long-term secrets---I
|
|
|
|
|
can easily change them.
|
|
|
|
|
But you could easily do both---
|
|
|
|
|
first require decryption with your smart card and then enter a passphrase
|
|
|
|
|
for a symmetric key.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2019-03-25 12:53:44 -04:00
|
|
|
|
*** REHEARSED Taking Screenshots :B_frame:
|
2019-03-16 23:45:57 -04:00
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: frame
|
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
#+BEGIN_SRC sh
|
|
|
|
|
# draw region on screen, output to ss.png
|
|
|
|
|
$ import ss.png
|
|
|
|
|
|
|
|
|
|
# screenshot of entire screen after 5 seconds
|
|
|
|
|
$ import -pause 5 -window root ss.png
|
|
|
|
|
|
|
|
|
|
# screenshot to clipboard
|
|
|
|
|
import png:- | xclip -i -selection clipboard -t image/png
|
|
|
|
|
#+END_SRC
|
|
|
|
|
|
2019-03-17 12:14:22 -04:00
|
|
|
|
**** Notes :B_noteNH:
|
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: noteNH
|
2019-03-20 00:01:10 -04:00
|
|
|
|
:DURATION: 00:00:35
|
2019-03-17 12:14:22 -04:00
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
Okay,
|
|
|
|
|
back to something less serious.
|
|
|
|
|
ImageMagick also comes with a vaguely-named utility called =import=,
|
|
|
|
|
which can capture images from an X server.
|
|
|
|
|
Screenshots.
|
|
|
|
|
|
|
|
|
|
If you run it with just a filename,
|
|
|
|
|
then it'll change your cursor to a cross.
|
|
|
|
|
You can drag to define a rectangular region to copy,
|
|
|
|
|
or you can just click on a window to take a screenshot of the whole thing.
|
|
|
|
|
|
2019-03-20 00:01:10 -04:00
|
|
|
|
The bottom example avoids outputting to a file entirely and instead feeds
|
2019-03-17 12:14:22 -04:00
|
|
|
|
the raw image data to standard out.
|
|
|
|
|
We then pipe that to =xclip= to copy it to the clipboard.
|
|
|
|
|
Notice the new =image/png= type there.
|
|
|
|
|
Now you can paste the image into other programs,
|
|
|
|
|
like image editors or websites.
|
|
|
|
|
|
2019-03-25 12:53:44 -04:00
|
|
|
|
*** REHEARSED Screenshot OCR :B_frame:
|
2019-03-16 23:45:57 -04:00
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: frame
|
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
#+BEAMER: \begin{onlyenv}<+>
|
|
|
|
|
#+BEGIN_SRC sh
|
|
|
|
|
$ import png:- | tesseract -psm 3 - - | cowsay
|
|
|
|
|
__________________________________
|
|
|
|
|
/ Keynote Speakers \
|
|
|
|
|
| |
|
|
|
|
|
| i |
|
|
|
|
|
| |
|
|
|
|
|
| Bdale Garbee Micky Metts Richard |
|
|
|
|
|
\ Stallman Tarek Loubani /
|
|
|
|
|
----------------------------------
|
|
|
|
|
\ ^__^
|
|
|
|
|
\ (oo)\_______
|
|
|
|
|
(__)\ )\/\
|
|
|
|
|
||----w |
|
|
|
|
|
|| ||
|
|
|
|
|
#+END_SRC
|
|
|
|
|
#+BEAMER: \end{onlyenv}
|
|
|
|
|
|
|
|
|
|
#+BEAMER: \begin{onlyenv}<+>
|
|
|
|
|
#+BEGIN_SRC sh
|
|
|
|
|
# OCR at URL
|
|
|
|
|
$ wget -qO- https://.../keynote_banner.png \
|
|
|
|
|
| tesseract -psm 3 - -
|
|
|
|
|
|
|
|
|
|
# perform OCR on selected region and speak it
|
|
|
|
|
$ import png:- | tesseract -psm 3 - - | espeak
|
|
|
|
|
|
|
|
|
|
# perform OCR on clipboard image and show in dialog
|
|
|
|
|
$ xclip -o -selection clipboard -t image/png \
|
|
|
|
|
| tesseract -psm 3 - - \
|
|
|
|
|
| xargs -0 zenity --info --text
|
|
|
|
|
#+END_SRC
|
|
|
|
|
#+BEAMER: \end{onlyenv}
|
|
|
|
|
|
2019-03-17 12:14:22 -04:00
|
|
|
|
**** Notes :B_noteNH:
|
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: noteNH
|
2019-03-20 00:01:10 -04:00
|
|
|
|
:DURATION: 00:01:00
|
2019-03-17 12:14:22 -04:00
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
Since =import= lets us select a region of the screen,
|
|
|
|
|
what if we used that to create a script that runs OCR on whatever we
|
|
|
|
|
select?
|
|
|
|
|
And just for fun,
|
|
|
|
|
what if the translation was presented to us by a cow?
|
|
|
|
|
|
2019-03-20 00:01:10 -04:00
|
|
|
|
=tesseract= is a free (as in freedom) optical character recognition
|
|
|
|
|
program.
|
|
|
|
|
|
2019-03-17 12:14:22 -04:00
|
|
|
|
In this example,
|
|
|
|
|
I selected the keynote speaker image on the LP2019 website.
|
|
|
|
|
It worked quite well.
|
|
|
|
|
|
|
|
|
|
But a cow isn't practical.
|
|
|
|
|
Instead,
|
|
|
|
|
you can imagine maybe binding this to a key combination in your window
|
|
|
|
|
manager.
|
|
|
|
|
In that case,
|
|
|
|
|
maybe it'd be more convenient if it was spoken to you,
|
|
|
|
|
like the second example.
|
|
|
|
|
Or maybe copied to your clipboard.
|
|
|
|
|
|
|
|
|
|
In the third example,
|
|
|
|
|
we take an image off of the clipboard,
|
|
|
|
|
run OCR,
|
|
|
|
|
and then display the resulting text in a GUI dialog with =zenity=.
|
|
|
|
|
|
|
|
|
|
I really like these examples,
|
|
|
|
|
because to an average user,
|
|
|
|
|
all of this seems somewhat novel.
|
|
|
|
|
And yet,
|
|
|
|
|
it's one of the simplest examples we've done!
|
|
|
|
|
|
2019-03-20 00:01:10 -04:00
|
|
|
|
YMMV though with tesseract,
|
2019-03-17 12:14:22 -04:00
|
|
|
|
depending on your settings and training data.
|
|
|
|
|
|
2019-03-25 12:53:44 -04:00
|
|
|
|
*** REHEARSED Full Circle :B_fullframe:
|
2019-03-16 23:45:57 -04:00
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: fullframe
|
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
#+BEGIN_SRC sh
|
|
|
|
|
$ xdotool search --name ' GNU IceCat$' windowactivate --sync \
|
2019-03-17 12:14:22 -04:00
|
|
|
|
windowsize 1024 600 \
|
|
|
|
|
key ctrl+t \
|
2019-03-16 23:45:57 -04:00
|
|
|
|
&& while read url; do
|
|
|
|
|
xdotool getactivewindow \
|
|
|
|
|
key ctrl+l type "$url" \
|
|
|
|
|
&& xdotool getactivewindow \
|
|
|
|
|
key Return sleep 5 \
|
|
|
|
|
key ctrl+f sleep 0.5 \
|
|
|
|
|
type 'free software' \
|
|
|
|
|
&& import -window "$( xdotool getactivewindow )" \
|
|
|
|
|
-crop 125x20+570+570 png:- \
|
|
|
|
|
| tesseract -psm 3 - - \
|
|
|
|
|
| grep -q ^Phrase && echo "$url" \
|
|
|
|
|
done < url-list | tee results.txt
|
|
|
|
|
#+END_SRC
|
|
|
|
|
|
2019-03-17 12:14:22 -04:00
|
|
|
|
**** Notes :B_noteNH:
|
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: noteNH
|
2019-03-20 00:01:10 -04:00
|
|
|
|
:DURATION: 00:02:25
|
2019-03-17 12:14:22 -04:00
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
Okay,
|
|
|
|
|
for this final example,
|
|
|
|
|
I'm going to do something a bit more complex.
|
|
|
|
|
Please take a few seconds to look at this.
|
|
|
|
|
<pause>
|
|
|
|
|
|
|
|
|
|
=xdotool= is a swiss army knife of X11 operations.
|
|
|
|
|
It can find windows,
|
|
|
|
|
give them focus,
|
|
|
|
|
send keystrokes,
|
|
|
|
|
move the mouse,
|
|
|
|
|
and do many other things.
|
|
|
|
|
|
|
|
|
|
The first line here finds the first window ending in ``GNU IceCat'' and
|
|
|
|
|
gives it focus.
|
|
|
|
|
=--sync= blocks until that completes so we don't proceed too hastily.
|
|
|
|
|
We then proceed to resize it to 1024x600,
|
|
|
|
|
and then open a new tab by sending =Ctrl+T=.
|
|
|
|
|
|
|
|
|
|
That's right.
|
|
|
|
|
We've come full circle back to the web browser.
|
|
|
|
|
|
|
|
|
|
We then proceed to read each URL form a file =url-list=,
|
|
|
|
|
which you can see after =done=.
|
|
|
|
|
For each URL,
|
|
|
|
|
we send =Ctrl+L= to give focus to the location bar and then type in the
|
|
|
|
|
URL.
|
|
|
|
|
We can't chain any commands in =xdotool= after =type=,
|
|
|
|
|
which is why we have another invocation to hit =Return=.
|
|
|
|
|
We then sleep for five seconds to give time to navigate.
|
|
|
|
|
I use Tor,
|
|
|
|
|
so latency varies.
|
|
|
|
|
We then send =Ctrl+F=,
|
|
|
|
|
wait very briefly to give IceCat time to trigger the find,
|
|
|
|
|
and then send our string to search for.
|
|
|
|
|
|
|
|
|
|
If you remember,
|
|
|
|
|
I mentioned that part of the problem with the GUI approach is that it
|
|
|
|
|
requires visual inspection.
|
|
|
|
|
Fortunately for us,
|
|
|
|
|
we have =tesseract=!
|
|
|
|
|
So we take a screenshot of the IceCat window at the offset where the match
|
|
|
|
|
results are on my system.
|
|
|
|
|
We pipe that image to =tesseract= and,
|
|
|
|
|
if it contains the word ``Phrase'',
|
|
|
|
|
as in ``Phrase not found'',
|
|
|
|
|
we echo the URL as a non-match.
|
|
|
|
|
All of this is =tee='d to =results.txt= like before.
|
|
|
|
|
|
|
|
|
|
This is /not/ what I had in mind when I talked about melding mind and
|
|
|
|
|
machine.
|
|
|
|
|
But this shows that,
|
|
|
|
|
even with GUIs,
|
|
|
|
|
we can produce some level of automation using existing tools and a little
|
|
|
|
|
bit of creativity.
|
|
|
|
|
Of course,
|
2019-03-20 00:01:10 -04:00
|
|
|
|
this may not work on your system if your font size is different or because
|
2019-03-17 12:14:22 -04:00
|
|
|
|
of various other factors;
|
|
|
|
|
to generalize this,
|
|
|
|
|
we'd have to get more creative to find the result text.
|
|
|
|
|
|
|
|
|
|
It's also worth mentioning that =xdotool= can come in handy for the password
|
|
|
|
|
manager too---instead
|
|
|
|
|
of copying to the clipboard,
|
|
|
|
|
we can type it directly into a window.
|
|
|
|
|
And by specifying which window to send it to,
|
|
|
|
|
we can ensure that we don't accidentally type in the wrong window,
|
|
|
|
|
like a chat,
|
|
|
|
|
if the user changes focus.
|
|
|
|
|
That also thwarts systems that implement the terrible anti-pattern of try to
|
|
|
|
|
prevent you from pasting passwords.
|
|
|
|
|
|
|
|
|
|
|
2019-03-25 12:53:44 -04:00
|
|
|
|
*** REHEARSED Getting Help :B_frame:
|
2019-03-16 23:45:57 -04:00
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: frame
|
|
|
|
|
:END:
|
|
|
|
|
|
2019-03-17 12:14:22 -04:00
|
|
|
|
- All GNU packages have Info manuals and =--help=
|
2019-03-16 23:45:57 -04:00
|
|
|
|
- Most programs (including GNU) have manpages
|
|
|
|
|
|
|
|
|
|
#+BEAMER: \subskip
|
|
|
|
|
|
|
|
|
|
#+BEGIN_SRC sh
|
2019-03-17 12:14:22 -04:00
|
|
|
|
$ grep --help # usage information for grep
|
|
|
|
|
$ man grep # manpage for grep
|
|
|
|
|
$ info grep # full manual for grep
|
2019-03-16 23:45:57 -04:00
|
|
|
|
|
|
|
|
|
# bash help
|
|
|
|
|
$ help
|
|
|
|
|
$ man bash
|
|
|
|
|
#+END_SRC
|
|
|
|
|
|
2019-03-17 12:14:22 -04:00
|
|
|
|
**** Notes :B_noteNH:
|
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: noteNH
|
2019-03-20 00:01:10 -04:00
|
|
|
|
:DURATION: 00:01:30
|
2019-03-17 12:14:22 -04:00
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
I have presented a lot of stuff here,
|
|
|
|
|
and I only got to a fraction of what I would have liked to talk about.
|
|
|
|
|
Some of you watching this aren't familiar with a lot of these topics,
|
|
|
|
|
or possibly /any/ of them,
|
|
|
|
|
so getting started may seem like a daunting task.
|
|
|
|
|
Where do you start?
|
|
|
|
|
|
|
|
|
|
As I showed before,
|
|
|
|
|
the process of writing a command line is an iterative one.
|
|
|
|
|
Learning is the same way.
|
|
|
|
|
I don't remember all of the options for these programs.
|
|
|
|
|
=xdotool= I had used only lightly before this talk,
|
|
|
|
|
for example;
|
|
|
|
|
I had to research how to write that command line.
|
|
|
|
|
But I didn't use the Internet to do it;
|
|
|
|
|
I did it from the comfort of my terminal using only what was already
|
|
|
|
|
installed on my system.
|
|
|
|
|
|
|
|
|
|
You can usually get usage information for programs by typing =--help= after
|
|
|
|
|
the command.
|
|
|
|
|
This will work for all GNU programs,
|
|
|
|
|
but not all command line programs implement it.
|
|
|
|
|
All GNU programs also offer Info manuals,
|
|
|
|
|
which read like books;
|
|
|
|
|
those can be found with the =info= command,
|
|
|
|
|
and can also be read in a much better format using Emacs.
|
|
|
|
|
You can also usually find them online in HTML format.
|
2019-03-20 00:01:10 -04:00
|
|
|
|
Most command line programs for Unix-like operating systems also include
|
|
|
|
|
manpages,
|
|
|
|
|
accessible with the =man= command.
|
2019-03-17 12:14:22 -04:00
|
|
|
|
If you want to know how to use =man=,
|
|
|
|
|
you can view its own manpage using =man man=.
|
|
|
|
|
|
|
|
|
|
You can also get some help using Bash itself by simply typing =help= to list
|
|
|
|
|
its builtins,
|
|
|
|
|
and it has a very comprehensive manpage.
|
|
|
|
|
|
|
|
|
|
This is obviously a bit different than how people interact with GUIs,
|
|
|
|
|
which are designed to be discoverable without the need for users to read
|
|
|
|
|
documentation.
|
|
|
|
|
|
|
|
|
|
|
2019-03-04 23:39:04 -05:00
|
|
|
|
** Thank You :B_fullframe:
|
|
|
|
|
:PROPERTIES:
|
|
|
|
|
:BEAMER_env: fullframe
|
|
|
|
|
:DURATION: 00:00:01
|
|
|
|
|
:END:
|
|
|
|
|
|
|
|
|
|
#+BEGIN_COMMENT
|
|
|
|
|
Thank you.
|
|
|
|
|
#+END_COMMENT
|
|
|
|
|
|
|
|
|
|
#+BEGIN_CENTER
|
|
|
|
|
Mike Gerwitz
|
|
|
|
|
|
|
|
|
|
[[mailto:mtg@gnu.org][=mtg@gnu.org=]]
|
|
|
|
|
|
|
|
|
|
\bigskip
|
|
|
|
|
|
|
|
|
|
Slides and Source Code Available Online
|
|
|
|
|
|
2019-03-25 12:53:44 -04:00
|
|
|
|
<[[https://mikegerwitz.com/talks/cs4m]].pdf>
|
2019-03-04 23:39:04 -05:00
|
|
|
|
|
|
|
|
|
\bigskip
|
|
|
|
|
|
|
|
|
|
\vfill
|
|
|
|
|
|
|
|
|
|
Licensed under the Creative Commons Attribution ShareAlike 4.0
|
|
|
|
|
International License
|
|
|
|
|
#+END_CENTER
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
* Exporting :noexport:
|
|
|
|
|
For a non-interactive build, including all dependencies, simply run =make=.
|
|
|
|
|
|
|
|
|
|
Once all dependencies are built, you should be able to simply export this
|
|
|
|
|
buffer as a Beamer presentation (=C-c C-e l P=) to get an updated PDF (or
|
|
|
|
|
you can just run =make= again).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
* Copyright and Licenses :noexport:
|
|
|
|
|
This file Copyright (C) 2019 Mike Gerwitz.
|
|
|
|
|
|
|
|
|
|
This file is licensed under the Creative Commons Attribution ShareAlike 4.0
|
|
|
|
|
International License. See [[file:COPYING.CCBYSA]] for the full license text.
|
|
|
|
|
|
|
|
|
|
Additionally, all code fragments are dual-licensed as such:
|
|
|
|
|
|
|
|
|
|
All code fragments are free software: you can redistribute it and/or modify
|
|
|
|
|
it under the terms of the GNU General Public License as published by
|
|
|
|
|
the Free Software Foundation, either version 3 of the License, or
|
|
|
|
|
(at your option) any later version.
|
|
|
|
|
|
|
|
|
|
This program is distributed in the hope that it will be useful,
|
|
|
|
|
but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
|
|
|
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
|
|
|
|
GNU General Public License for more details.
|
|
|
|
|
|
|
|
|
|
You should have received a copy of the GNU General Public License
|
|
|
|
|
along with this program (see [[file::COPYING]]). If not, see
|
|
|
|
|
<http://www.gnu.org/licenses/>.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
* Local Variables :noexport:
|
|
|
|
|
# Local Variables:
|
|
|
|
|
# org-todo-keyword-faces: (("DRAFT" . org-upcoming-deadline) \
|
|
|
|
|
# ("DEVOID" . (:inherit org-warning \
|
|
|
|
|
# :inverse-video t)) \
|
|
|
|
|
# ("LACKING" . org-warning) \
|
|
|
|
|
# ("REVIEWED" . "yellow") \
|
|
|
|
|
# ("AUGMENT" . (:foreground "yellow" :bold t :underline t)) \
|
|
|
|
|
# ("READY" . (:inherit org-scheduled :bold t :underline t)))
|
|
|
|
|
# eval: (add-to-list 'org-structure-template-alist
|
|
|
|
|
# '("C" "#+BEGIN_COMMENT\n?\n#+END_COMMENT"))
|
|
|
|
|
# End:
|