#apue — Public Fediverse posts
Live and recent posts from across the Fediverse tagged #apue, aggregated by home.social.
-
Hey kids! It's the start of the Fall semester, and I'm again teaching "Advanced Programming in the UNIX Environment".
The syllabus and all course materials including all code examples are available here:
https://stevens.netmeister.org/631/As usual, we'll be using #NetBSD as our main platform. All video lectures are public and available for free on YouTube:
https://www.youtube.com/playlist?list=PL0qfF8MrJ-jxMfirAdxDs9zIiBg2Wug0zI'll be posting individual lecture videos and related links in this thread throughout the semester.
-
Advanced Programming in the Unix Environment
The entire semester in one 157 slides deck: https://stevens.netmeister.org/631/f25.pdf
Full playlist on youtube of all 72 video lectures: https://www.youtube.com/watch?v=BsB9Cg6yJc4&list=PL0qfF8MrJ-jxMfirAdxDs9zIiBg2Wug0z
-
Advanced Programming in the Unix Environment
Week 13: Restricting Processes: Capabilities, Control Groups, Containers
In this video lecture, we're completing our discussion on how to restrict processes by looking at POSIX Capabilities, Linux Namespaces and Control Groups or "cgroups", and how these and the various other methods we've discussed in the last few videos allow us build containers like Docker or LXC.
-
Advanced Programming in the Unix Environment
Week 13: Restricting Processes: Processor Affinity and CPU Sets
In this video lecture, we discuss how we can restrict a process to a specific CPU or set of CPUs by way of "CPU pinning" or processor affinity and via CPU sets.
This lecture also exists as a blog post version as well:
https://www.netmeister.org/blog/cpu-pinning-and-sets.html -
Advanced Programming in the Unix Environment
Week 13: Restricting Processes: Process Priorities
In this video lecture, we revisit resource utilization by processes and how to set their CPU scheduling priority or "niceness". We discuss dynamic priority scheduling algorithms and the use of the nice(1) and renice(8) utilities.
-
Advanced Programming in the Unix Environment
Week 13: Restricting Processes: Restricted shells, Chroots, Jails
In this video lecture, we cover restricted shells, the chroot(2) syscall, and FreeBSD jails as ways to contain users and processes beyond file access controls.
-
Advanced Programming in the Unix Environment
Week 13: Restricting Processes: eUIDs, file flags, mount options, securelevels
In this video lecture, we look at new ways to restrict processes and even root itself. We revisit how we change effective UIDs using setuid, su(1) and sudo(8), and then cover file flags, certain mount options, and BSD securelevels.
-
Advanced Programming in the Unix Environment
Week 13: Restricting Processes: POSIX ACLs
This week we're diving deeper into ways to restrict processes. We begin with a review of standard Unix file permissions and then show how POSIX.1e Access Control Lists (ACLs) can be used for more fine-grained control.
-
Advanced Programming in the Unix Environment
Week 12: Cryptographic Basics
A segment on where and how cryptography can help solve certain security issues, including a few code examples for how to handle secrets/passwords (and how they linger in memory!), how to hash passwords, and the impact of poor pseudo-random number generators (with a blast from the past via the 2008 Debian weak keys).
Lecture slides:
https://stevens.netmeister.org/631/12-cryptobasics.pdf -
Advanced Programming in the Unix Environment
Week 12: Asynchronous and Memory Mapped I/O
With this video, we conclude our short series on advanced I/O topics with a quick summary of asynchronous (aio(7)) and memory mapped I/O (mmap(2)).
-
Advanced Programming in the Unix Environment
Week 12: Resource Locking
More miscellaneous "advanced I/O" - this time we discuss resource- and record locking using flock(2), lockf(3), and fcntl(2).
-
Advanced Programming in the Unix Environment
Week 12: Non-blocking I/O
Continuing our grab bag of miscellaneous topics, we briefly cover non-blocking I/O, whereby a system call that would normally block, waiting for I/O, returns immediately with an errno of EAGAIN or EWOULDBLOCK. We use the examples of I/O into a pipe (restricted to PIPE_BUF bytes at a time) or over the network using TCP with its buffer.
-
Advanced Programming in the Unix Environment
Week 12: syslog(3)
In this video lecture, we take a look at the standard Unix central logging facility, syslogd(8), and the syslog(3) library function. We show examples of system logs and how to separate messages by priority and level.
Below is an accurate capture of syslogd(8) in action:
-
Advanced Programming in the Unix Environment
I often see students confuse the "sizeof" operator and the "strlen" library function, especially when mixing fixed-sized buffers vs. pointers. So as a brief intermezzo, let's explain the difference between the two.
Based on that, I eventually ended up writing a manual page for 'sizeof', which is now included in #FreeBSD:
-
Advanced Programming in the Unix Environment
Week 11: Of Linkers and Loaders
Picking up where we left off when we talked about the compilation and linking process in Week 05, we now take a look at how a relocatable object file is turned into an executable by the dynamic linker (ld) and how an executable is loaded into memory by the run-time link-editor (ld.so or ld.elf_so).
-
Advanced Programming in the Unix Environment
Week 11: The Executable and Linkable Format
This week, we begin our exploration of shared libraries. We start with a look at the Executable and Linkable Format (ELF) for binary files such as executables, object files, core files and shared libraries. We'll use the hexdump(1) and readelf(1) utilities to better understand the format.
-
Advanced Programming in the Unix Environment
Week 10: Dæmon Processes
In this video lecture, we take a quick look at processes intended to run continuously and in the background: daemon processes. We'll also brush upon system start scripts and daemon/service conventions.
-
Advanced Programming in the Unix Environment
Week 9: I/O Multiplexing
With this video, we complete our discussion of interprocess communication and move from one-to-one communications towards a more typical client-server model with a server side process capable of handling multiple simultaneous clients. We accomplish this by using the select(2) syscall to handle synchronous I/O multiplexing.
-
Advanced Programming in the Unix Environment
Week 9: socket(PF_INET6, SOCK_STREAM, 0)
In this video, we demonstrate sockets using TCP to establish a sequenced, reliable, two-way byte stream over an #IPv6 network. Our good friend tcpdump(8) helps us inspect the packets on the wire to observe the 3-way handshake, data being exchanged, and connection tear down or observing a reset (RST) packet from the server if it's not listening on the port.
-
Advanced Programming in the Unix Environment
Week 9: socket(PF_INET, SOCK_DGRAM, 0)
We're finally moving on to network communications, and show how to communicate between hosts across the internet using datagram sockets in the PF_INET domain, that is, via UDP. We also observe the packets as they are sent by using the tcpdump(8) utility.
-
Advanced Programming in the Unix Environment
Week 9: socket(PF_LOCAL, SOCK_DGRAM, 0)
Following up on socketpairs, we're continuing our discussion of the sockets API by an example of datagram connections in the PF_LOCAL (formerly AF_UNIX) domain.
-
Advanced Programming in the Unix Environment
Week 9: socketpair(2)
Let's continue with the above mentioned IPC tutorial and discuss the socketpair(2) system call and compare its functionality with that of the pipe(2) call. This introduces the concept of sockets, domains, and protocols, which we'll dive into in the next few videos.
-
Advanced Programming in the Unix Environment
Did you know that #NetBSD comes with a number of historical Unix research papers and supplementary documentation? Here, we find papers by Marshall Kirk McKusick on the Fast File System, by Robert Morris and Ken Thompson on Password Security, a shell tutorial by Stephen R. Bourne, a guide to using vi(1) by Bill Joy, and the well known BSD IPC Tutorials! Take a tour through /usr/share/doc...
-
Advanced Programming in the Unix Environment
Week 8: Pipes and FIFOs
With this video lecture, we continue our discussion of Interprocess Communication and dive into two of the oldest and most ubiquitous forms of Unix IPC: pipes and FIFOs. We also discuss popen(3) and the dangers of shelling out to unverified commands.
-
Advanced Programming in the Unix Environment
Week 8: System V IPC
In this video lecture, we cover traditional, primarily asynchronous System V Inter Process Communication: semaphores, shared memory, and message queues. We then also take a brief look at the POSIX message queues implementation.
-
Advanced Programming in the Unix Environment
Week 8: Interprocess Communications Intro
This week, we begin our discussion of Interprocess Communication. We start by providing a quick overview of the different properties of IPC mechanisms as well as which mechanisms we will cover.
-
Advanced Programming in the Unix Environment
Week 7: Reentrant and Interrupted Functions
With this video lecture, we will conclude our coverage of Unix signals, by looking at what happens when we call unsafe functions from within the signal handler as well as what happens when we are interrupted while blocking on certain I/O.
-
Advanced Programming in the Unix Environment
Week 7: Signals
In this video lecture, we look at Unix signals, simple, asynchronous event notifications. We run through a number of examples to illustrate how signals are delivered, can be ignored, caught, handled, or blocked.
-
Advanced Programming in the Unix Environment
Week 7: Job Control
In this video lecture, we illustrate the concept of job control in the shell, first introduced in the C shell, and allowing you to run multiple tasks from within the same terminal, switching back and forth between them by placing them into the background, suspending them, or bringing them to the foreground. It's one of my all-time favorite productivity hacks - Ctrl+Z FTW!
-
Advanced Programming in the Unix Environment
Week 7: Process Groups and Sessions
Let's take a look at how processes are grouped together and begin to develop an understanding of how a login session relates to the controlling terminal.