#+startup: beamer #+TITLE: Computational Symbiosis: Methods That Meld Mind and Machine #+AUTHOR: Mike Gerwitz #+EMAIL: mtg@gnu.org #+DATE: LibrePlanet 2019 #+BEAMER_HEADER: \title{Computational Symbiosis} #+BEAMER_HEADER: \subtitle{Methods That Meld Mind and Machine} #+OPTIONS: H:3 num:nil toc:nil p:nil todo:nil stat:nil #+LaTeX_CLASS: beamer #+LaTeX_CLASS_OPTIONS: [presentation] #+BEAMER_THEME: Madrid #+BEAMER_COLOR_THEME: beetle #+BEAMER_HEADER: \input{slides-preamble.tex} #+TODO: RAW(r) DEVOID(v) LACKING(l) DRAFT(d) REVIEWED(R) AUGMENT(A) | READY(,) REHEARSED(.) #+COLUMNS: %40ITEM %10DURATION{:} %8TODO %BEAMER_ENV(ENVIRONMENT) * Project Notes :noexport: ** Core Guidelines - Blur distinction between user and programmer. - Use only examples that non-programmers could come up with and understand. - GUIs /are/ sometimes the most efficient means of melding mind and machine. - De-emphasize doing everything on a terminal just because its "cool". ** Concepts [5/27] - [ ] =/sys= and such - [ ] Automating multi-step workflows with GNU Make - [ ] Concurrency - [ ] Connecting graphical and command line (X11 integration) [0/3] - [ ] Clipboard (image) - [ ] Clipboard (text) - [ ] X11 commands - [ ] Discoverability - Example: I used GIMP for editing some images for this talk because I thought it'd be faster to discover how to that way vs. ImageMagick docs. - [ ] Editor should keep up with your thoughts - Emphasize muscle memory - [-] Efficiency gains of light use of keybindings [2/3] - [X] Seldom change ([[*Browser Topics][Browser Topics]]) - [X] Some are universal (between programs) ([[*Browser Topics][Browser Topics]]) - [ ] Some context-sensitive and Just Work without having to hunt through context-specific menus - [ ] Emphasize free software community and how OS is built by so many contributors - Symbiosis is not only with machine, but community - [ ] Globbing - [ ] GnuPG and smart cards - [ ] Incremental development [0/3] - [ ] Develop as you go, perhaps just referencing history early on - [ ] Evolve by making portions of command dynamic (variables, subshells) - [ ] Transfer commands from history into scripts and aliases for re-use - [ ] Interactive prototyping - [ ] Looping - [ ] McIlroy and Knuth - [X] Mouse has dependency on positioning of UI elements ([[*Browser Topics][Browser Topics]]) - Changes over time, subject to user/system preferences, etc - [ ] Multiplexing with GNU Screen - [ ] Output redirection - [ ] Phone as an example of a device that best works with touch and GUIs and sacrifices practical freedom. - Requires specialized knowledge and is inconvenient to work with. - [X] Pipelines instead of intermediate files ([[*Browser Topics][Browser Topics]]). - [ ] Practically exercising software freedom [0/2] - [ ] Users need to be able to convey their thoughts to the computer without being programmers - [ ] Concise primitives / building blocks - [ ] Readline (history and editing) - [X] Regular expressions ([[*Browser Topics][Browser Topics]]) - [ ] Remote commands via SSH - [X] Text as a universal interface - [X] All programs can work with one-another ([[*Browser Topics][Browser Topics]]) - [X] Can replace any part of the process with a human---no difference between input from a program vs. from a keyboard. ([[*Browser Topics][Browser Topics]]) - [X] Using keybindings in a GUI with no mouse ([[*Browser Topics][Browser Topics]]) - Including transferring data between programs - [ ] Using macros to script keypresses (Vim) - [ ] Version control ** Programs [0/4] - [-] Coreutils / Common [3/12] - [ ] =column= - [ ] =diff= - [X] =echo= ([[*Browser Topics][Browser Topics]]). - [ ] =find= - [ ] =gpg= - [X] =grep= ([[*Browser Topics][Browser Topics]]) - [ ] =make= / Makefiles - [ ] =sort= - [ ] =ssh= - [ ] =uniq= - [ ] =wc= - [X] =wget= / =curl= ([[*Browser Topics][Browser Topics]]) - [ ] More Sophisticated [0/3] - [ ] =awk= - [ ] =git= - [ ] =sed= - [ ] X11 / Graphical [0/5] - [ ] =convert= - [ ] =i3= / =xmonad= / etc - [ ] =import= - [ ] =setxkbmap= - [ ] =xclip= / =xsel= - [ ] Editors / Multiplexing [0/4] - [ ] =emacs= - [ ] =nano= - [ ] =vim= - [ ] =screen= * DEVOID Slides [0/4] :PROPERTIES: :ID: slides :END: ** Summary :noexport: #+BEGIN: columnview :hlines 2 :maxlevel 3 :indent t :id slides | ITEM | DURATION | TODO | ENVIRONMENT | |---------------------------------------------------+----------+--------+-------------| | Slides | 1:00:00 | DEVOID | | |---------------------------------------------------+----------+--------+-------------| | \_ Summary | | | | |---------------------------------------------------+----------+--------+-------------| | \_ Introduction | | RAW | note | |---------------------------------------------------+----------+--------+-------------| | \_ Choreographed Workflows | | DEVOID | fullframe | |---------------------------------------------------+----------+--------+-------------| | \_ Practical Freedom | | DEVOID | fullframe | |---------------------------------------------------+----------+--------+-------------| | \_ Practical Example: Web Browser | 1:00 | DEVOID | | | \_ Browser Topics | | | | | \_ Example: Web Browser | 1:00 | RAW | frame | | \_ Finding Text (Mouse-Driven GUI Interaction) | | RAW | frame | | \_ GUIs Change Over Time | | RAW | frame | |---------------------------------------------------+----------+--------+-------------| | \_ Thank You | 00:00:01 | | fullframe | #+END: ** RAW Introduction :B_note: :PROPERTIES: :BEAMER_env: note :END: Hello, everyone! My name is Mike Gerwitz. I am a free software hacker and activist with a focus on user privacy and security. I'm also a GNU Maintainer and software evaluator, and hold other various other administrative duties within GNU. I have about twenty years of programming experience, half of that professionally, And I've been a computer user for longer. So I've been around long enough to see a decent evolution in how we interact with computers. I've gotten a sense of what feel right and wrong as both a user and a hacker. And interestingly, what I've settled on for my computing is really a toolset that was devised decades before I was even born, with some light modernization. And those tools don't work for everyone. But I think a subset of them can. So I'm here today to try to explore a healthy balance, and walk you through what I see as an efficient means of computing, based on the problems that I've encountered, and the problems I've seen others encounter over the years. ** DEVOID Choreographed Workflows :B_fullframe: :PROPERTIES: :BEAMER_env: fullframe :END: #+BEAMER: \fullslidetext Choreographed Workflows ** DEVOID Practical Freedom :B_fullframe: :PROPERTIES: :BEAMER_env: fullframe :END: #+BEAMER: \fullslidetext Practical Freedom ** DEVOID Practical Example: Web Browser [0/3] *** Browser Topics [2/7] :noexport: This example is the main segue from GUIs into the utility of keybindings into shell, so it has to be compelling. I chose something that is frequently done by users: visiting webpages and searching for text. - [X] Most webpages include styling, images, and other media that are best represented in a GUI. - Of course, I'm a firm believer in progressive enhancement, but this isn't a web development talk. - [X] Is it good for finding a word or phrase on a page? [4/4] - [X] Menu -> Find in This Page -> Type. Sure. - [X] Instructing a user how to do this is inherently visual. Comment on my need to highlight areas of screenshots. - [X] More experienced users may prefer =C-f=. - Emphasize the motion with my hand. Explain how that mental association is made. Really focus on this. - [X] Location of menu has changed over time (show screenshots), but =C-f= has been unchanged for decades. - [ ] What if I had a list of five webpages and I wanted to get rid of the ones that didn't have that word? [0/5] - [ ] Mouse and GUI way: - Open each page in a new tab. - Menu -> Find on page -> Type. - Found, copy URL, open a word processor, paste. - Go back, click X on the tab. - Repeat. - [ ] Then show keybinding way: - *Demonstrate this on the LP laptop.* - Open programs with =M-F2= - Open each page in a new tab (=C-t=) - =C-f ESC= - =C-l C-c= - =M-TAB C-v RET= - =M-TAB C-w= - Repeat - [ ] Emphasize the profound differences between these approaches - [ ] This would work with nearly /any/ combination of web browser and text editor. - [ ] But what if I had 10? 100? 1000? This is still tedious and slow. - What if I could walk away and get a coffee, play with the kids, come back and have it done for me? - Emphasize how users familiar with the tools I haven't yet mentioned may know how to do this, but people who don't know of Unix tools will still be thinking in terms of the UI, trying to figure out how to automate it. - [ ] Right-click on page and View Source - Briefly mention HTML and =C-f=, show how the text is on the page. - [ ] Open a terminal [0/4] - [ ] Type =wget URL= - [ ] Show how it downloaded a file, useful for offline viewing - [ ] Open it in a text editor, show how the text is there. =C-f=. - We eliminated the web browser and can still find the text /using the same keybinding/. - [ ] =grep word file.html= - We have just eliminated the editor. - [ ] Eliminate the temporary file entirely using a pipe - Show both =wget -O-= and =curl=. - [ ] Now script discovering what pages contain a certain word [0/4] - [ ] Mention previous example of being emailed a list of URLs. Rather than pasting them into a file, let's discover them using the same tool: =grep=. Save email to a file. - [ ] =grep -o 'https\?://[^ ]\+'=, show how it gets a list of URLs. - Mention how powerful regexes are, e.g. "find all the phone numbers on a page". - [ ] Introduce =while= and =read=, showing how we can iteratively run commands, maybe =... | while read url; do echo "URL: $url"; done=. - [ ] Introduce =-q= flag for grep, and use it to conditionally output only the URLs at which the word can be found. - Not enough talk time to walk away and get a coffee, but let's see if we can maybe let it run for 10--30s while I blabber on. Depends on the connection speed at MIT with all the participants. *** RAW Example: Web Browser :B_frame: :PROPERTIES: :BEAMER_env: frame :DURATION: 1:00 :END: [[file:images/web-browser.png]] **** Notes :B_noteNH: :PROPERTIES: :BEAMER_env: noteNH :END: One of the only GUIs I use on a day-to-day basis is my web browser. In this case, GNU Icecat, which is a Firefox derivative. This is a screenshot of an admittedly staged session, and contains a number of addons. Perhaps most prominent is Tree-Style Tab, which displays tabs in a hierarchy off to the left rather than flatly at the top. I find this visual enhancement very useful when I have dozens or hundreds of tabs open. There are text-mode web browsers, and I do occasionally use them. But it provides a very different experience. I don't want to spend too much time on my rationale, which as a professional web developer and activist focused on privacy and security run quite deep. For the sake of this talk, let's just recognize that most people who browse the internet use a graphical web browser; that's a simple fact. I chose a web browser as an example because I feel that it's something that most everyone can relate to using, and most everyone can recognize the utility in---most people using Internet-connected devices use one at least a few times a week, if not every day. *** RAW Finding Text (Mouse-Driven GUI Interaction) :B_frame: :PROPERTIES: :BEAMER_env: frame :END: **** Images :B_columns: :PROPERTIES: :BEAMER_env: columns :END: ***** Left :B_column: :PROPERTIES: :BEAMER_env: column :BEAMER_col: 0.30 :END: #+ATTR_LATEX: :width 1.5in [[file:images/ff-find-menu.png]] ***** Right :B_column: :PROPERTIES: :BEAMER_env: column :BEAMER_col: 0.70 :END: #+ATTR_LATEX: :width 3in [[file:images/ff-find-matches.png]] **** Notes :B_noteNH: :PROPERTIES: :BEAMER_env: noteNH :END: The Web is used for many different things today, but its original purpose is to render documents. Take Wikipedia for instance. Or the LibrePlanet conference website. If you are looking for something specific on a page, a common operation is to search for a word or phrase, like shown here. Now, how exactly to do this with a mouse varies depending on what program you're using, but here I highlighted the steps in a modern Icecat or Firefox. You start by clicking on the little hamburger, hotdog, or whatever-you-want-to-call-it menu in the upper-right, and then click on "Find in This Page" within the popup. This then opens a bar at the bottom of the page with an area to type the word or phrase you're searching for. It highlights and scrolls to the first match as you type, and has a button to highlight all results. It also shows the number of results off to the right. It's a simple, yet powerful mechanism that is pretty easy to use. So does a GUI provide the right tool for the job? If you're just searching for a name or concept, sure, that seems to be true. A GUI _is_ useful here. But notice how I had to convey these steps to you. I had to take screenshots and highlight where to click with the mouse. Since a GUI is inherently very visual, so are the instructions on how to use it. There is no canonical representation for these instructions, because it involves clicking on elements that have no clear name to the user. Unless you're in software or UX development, you may not know what to call that menu in the upper-right. Further, what do you call the bar at the bottom of the page? You have to describe it in a way that reproduces what the user sees. *** RAW GUIs Change Over Time :B_frame: :PROPERTIES: :BEAMER_env: frame :END: **** Images :B_columns: :PROPERTIES: :BEAMER_env: columns :END: ***** Left :B_column: :PROPERTIES: :BEAMER_env: column :BEAMER_col: 0.50 :END: #+ATTR_LATEX: :width 1.5in [[file:images/ff-find-menu.png]] ***** Right :B_column: :PROPERTIES: :BEAMER_env: column :BEAMER_col: 0.50 :END: #+ATTR_LATEX: :width 1.5in [[file:images/ff-edit-find.png]] #+BEAMER: \uncover<2>{\Large Ctrl+F #+BEAMER: } **** Notes :B_noteNH: :PROPERTIES: :BEAMER_env: noteNH :END: Another difficult thing is: GUIs change over time. I'm sure there are people here who remember earlier versions of Firefox that didn't have the hamburger menu, where the Find menu option was in the Edit menu. By the way, those old menus do still exist if you hit Alt. I miss the old menus, personally, because it /did/ make it easier to convey actions in text. Saying "Go to Edit - Find" is pretty clear, and those menu positions were always in the same place across the entire desktop environment. Now individual programs may vary in the their user experience. But do you notice something in common between these two screenshots? There's something that /hasn't/ changed over time---something that has been the same for /decades/! =Ctrl+F=. *** Ctrl+F---Just Works :B_frame: :PROPERTIES: :BEAMER_env: frame :END: - Most GUI programs that offer search - Context-sensitive---Do The Right Thing **** Notes :B_noteNH: :PROPERTIES: :BEAMER_env: noteNH :END: When you type =Ctrl+F=, it immediately opens that search bar and gives focus to the textbox, so you can just start typing. Further, it works /in any browser/. Not only that, but =Ctrl+F= is so universal that it works /in nearly every GUI program that offers some type of search/! And it's /context-sensitive/! The program will just Do The Right Thing depending on where you are or what you're doing. *** Muscle Memory :B_fullframe: :PROPERTIES: :BEAMER_env: fullframe :END: #+BEAMER: \fullslidetext Muscle Memory #+BEAMER: \fullsubtext Visual \Rightarrow Tactile **** Notes :B_noteNH: :PROPERTIES: :BEAMER_env: noteNH :END: But there's something more profound that has happened here, that many users don't even think about. We have switched our mode of interaction. With a mouse and a GUI, interaction is driven by visual indicators. The position of your hand isn't meaningful, because your mouse cursor could be anywhere on the screen at any given time; your eyes provide the context. It's hard to use a GUI with your eyes closed. But by hitting =Ctrl+F=, we've completely changed how we interact with the system. It's now /tactile/. You associate a finger placement; a motion; and the feeling of the keys being pressed beneath your fingers; with an action---finding something. You develop muscle memory. You _can_ trigger this feature with your eyes closed. /(Repeatedly make motion with hand and fingers like a madman during the above paragraph.)/ But that's a pretty trivial example. ** Thank You :B_fullframe: :PROPERTIES: :BEAMER_env: fullframe :DURATION: 00:00:01 :END: #+BEGIN_COMMENT Thank you. #+END_COMMENT #+BEGIN_CENTER Mike Gerwitz [[mailto:mtg@gnu.org][=mtg@gnu.org=]] \bigskip Slides and Source Code Available Online <[[https://mikegerwitz.com/talks/cs4m]]> \bigskip \vfill Licensed under the Creative Commons Attribution ShareAlike 4.0 International License #+END_CENTER * Exporting :noexport: For a non-interactive build, including all dependencies, simply run =make=. Once all dependencies are built, you should be able to simply export this buffer as a Beamer presentation (=C-c C-e l P=) to get an updated PDF (or you can just run =make= again). * Copyright and Licenses :noexport: This file Copyright (C) 2019 Mike Gerwitz. This file is licensed under the Creative Commons Attribution ShareAlike 4.0 International License. See [[file:COPYING.CCBYSA]] for the full license text. Additionally, all code fragments are dual-licensed as such: All code fragments are free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program (see [[file::COPYING]]). If not, see . * Local Variables :noexport: # Local Variables: # org-todo-keyword-faces: (("DRAFT" . org-upcoming-deadline) \ # ("DEVOID" . (:inherit org-warning \ # :inverse-video t)) \ # ("LACKING" . org-warning) \ # ("REVIEWED" . "yellow") \ # ("AUGMENT" . (:foreground "yellow" :bold t :underline t)) \ # ("READY" . (:inherit org-scheduled :bold t :underline t))) # eval: (add-to-list 'org-structure-template-alist # '("C" "#+BEGIN_COMMENT\n?\n#+END_COMMENT")) # End: