BSD Now

We read FreeBSD’s third quarterly status report, OpenBSD on Sparc64, ZoL repo move to OpenZFS, GEOM NOP, keeping NetBSD up-to-date, and more. Headlines FreeBSD third quarterly status report for 2019 This quarter the reports team has been more active than usual thanks to a better organization: calls for reports and reminders have been sent regularly, reports have been reviewed and merged quickly (I would like to thank debdrup@ in particular for his reviewing work). Efficiency could still be improved with the help of our community. In particular, the quarterly team has found that many reports have arrived in the last days before the deadline or even after. I would like to invite the community to follow the guidelines below that can help us sending out the reports sooner. Starting from next quarter, all quarterly status reports will be prepared the last month of the quarter itself, instead of the first month after the quarter's end. This means that deadlines for submitting reports will be the 1st of January, April, July and October. Next quarter will then be a short one, covering the months of November and December only and the report will probably be out in mid January. OpenBSD on Sparc64 OpenBSD, huh? Yes, I usually write about FreeBSD and that’s in fact what I tried installing on the machine first. But I ran into problems with it very early on (never even reached single user mode) and put it aside for later. Since I powered up the SunFire again last month, I needed an OS now and chose OpenBSD for the simple reason that I have it available. First I wanted to call this article simply “OpenBSD on SPARC” – but that would have been misleading since OpenBSD used to support 32-bit SPARC processors, too. The platform was just put to rest after the 5.9 release. Version 6.0 was the last release of OpenBSD that came on CD-ROM. When I bought it, I thought that I’d never use the SPARC CD. But here was the chance! While it is an obsolete release, it comes with the cryptographic signatures to verify the next release. So the plan is to start at 6.0 as I can trust the original CDs and then update to the latest release. This will also be an opportunity to recap on some of the things that changed over the various versions. News Roundup ZoL repo move to OpenZFS Because it will contain the ZFS source code for both Linux and FreeBSD, we will rename the "ZFSonLinux" code repository to "OpenZFS". Specifically, the repo at http://github.com/ZFSonLinux/zfs will be moved to the OpenZFS organization, at http://github.com/OpenZFS/zfs. The next major release of ZFS for Linux and FreeBSD will be "OpenZFS 2.0", and is expected to ship in 2020. Mcclure111 Sun Thread A long time ago— like 15 years ago— I worked at Sun Microsystems. The company was nearly dead at the time (it died a couple years later) because they didn't make anything that anyone wanted to buy anymore. So they had a lot of strange ideas about how they'd make their comeback. GEOM NOP Sometimes while testing file systems or applications you want to simulate some errors on the disk level. The first time I heard about this need was from Baptiste Daroussin during his presentation at AsiaBSDCon 2016. He mentioned how they had built a test lab with it. The same need was recently discussed during the PGCon 2019, to test a PostgreSQL instance. If you are FreeBSD user, I have great news for you: there is a GEOM provider which allows you to simulate a failing device. GNOP allows us to configure transparent providers from existing ones. The first interesting option of it is that we can slice the device into smaller pieces, thanks to the ‘offset option’ and ‘stripsesize’. This allows us to observe how the data on the disk is changing. Let’s assume that we want to observe the changes in the GPT table when the GPT flags are added or removed (for example the bootme flags which are described here). We can use dd every time and analyze it using absolute values from the disks. Keeping NetBSD up-to-date with pkg_comp 2.0 This is a tutorial to guide you through the shiny new pkg_comp 2.0 on NetBSD. Goals: to use pkg_comp 2.0 to build a binary repository of all the packages you are interested in; to keep the repository fresh on a daily basis; and to use that repository with pkgin to maintain your NetBSD system up-to-date and secure. This tutorial is specifically targeted at NetBSD but should work on other platforms with some small changes. Expect, at the very least, a macOS-specific tutorial as soon as I create a pkg_comp standalone installer for that platform. Beastie Bits DragonFly - Radeon Improvements NomadBSD review Spongebob OpenBSD Security Comic Forth : The Early Years LCM+L PDP-7 booting and running UNIX Version 0 Feedback/Questions Chris - Ctrl-T Improved Ctrl+t that shows kernel backtrace Brian - Migrating NexentaStore to FreeBSD/FreeNAS Avery - How to get involved Send questions, comments, show ideas/topics, or stories you want mentioned on t

LPI releases BSD Certification, openzfs trip report, Using FreeBSD with ports, LLDB threading support ready, Linux versus Open Source Unix, and more. Headlines Linux Professional Institute Releases BSD Specialist Certification - re BSD Certification Group Linux Professional Institute extends its Open Technology certification track with the BSD Specialist Certification. Starting October 30, 2019, BSD Specialist exams will be globally available. The certification was developed in collaboration with the BSD Certification Group which merged with Linux Professional Institute in 2018. G. Matthew Rice, the Executive Director of Linux Professional Institute says that "the release of the BSD Specialist certification marks a major milestone for Linux Professional Institute. With this new credential, we are reaffirming our belief in the value of, and support for, all open source technologies. As much as possible, future credentials and educational programs will include coverage of BSD.” OpenZFS Trip Report The seventh annual OpenZFS Developer Summit took place on November 4th and 5th in San Francisco and brought together a healthy mix of familiar faces and new community participants. Several folks from iXsystems took part in the talks, hacking, and socializing at this amazing annual event. The messages of the event can be summed up as Unification, Refinement, and Ecosystem Tooling. News Roundup Using FreeBSD with Ports (2/2): Tool-assisted updating Part 1 here: https://eerielinux.wordpress.com/2019/08/18/using-freebsd-with-ports-1-2-classic-way-with-tools/ In the previous post I explained why sometimes building your software from ports may make sense on FreeBSD. I also introduced the reader to the old-fashioned way of using tools to make working with ports a bit more convenient. In this follow-up post we’re going to take a closer look at portmaster and see how it especially makes updating from ports much, much easier. For people coming here without having read the previous article: What I describe here is not what every FreeBSD admin today should consider good practice (any more)! It can still be useful in special cases, but my main intention is to discuss this for building up the foundation for what you actually should do today. LLDB Threading support now ready for mainline Upstream describes LLDB as a next generation, high-performance debugger. It is built on top of LLVM/Clang toolchain, and features great integration with it. At the moment, it primarily supports debugging C, C++ and ObjC code, and there is interest in extending it to more languages. In February, I have started working on LLDB, as contracted by the NetBSD Foundation. So far I've been working on reenabling continuous integration, squashing bugs, improving NetBSD core file support, extending NetBSD's ptrace interface to cover more register types and fix compat32 issues and fixing watchpoint support. Then, I've started working on improving thread support which is taking longer than expected. You can read more about that in my September 2019 report. So far the number of issues uncovered while enabling proper threading support has stopped me from merging the work-in-progress patches. However, I've finally reached the point where I believe that the current work can be merged and the remaining problems can be resolved afterwards. More on that and other LLVM-related events happening during the last month in this report. Linux VS open source UNIX Beastie Bits Support for Realtek RTL8125 2.5Gb Ethernet controller Computer Files Are Going Extinct FreeBSD kernel hacking Modern BSD Computing for Fun on a VAX! Trying to use a VAX in today's world by Jeff Armstrong MidnightBSD 1.2 Released Feedback/Questions Paulo - Zfs snapshots Phillip - GCP A Listener - Old episodes? Send questions, comments, show ideas/topics, or stories you want mentioned on the show to [email protected] Your browser does not support the HTML5 video tag.

FreeBSD 12.1 is here, A history of Unix before Berkeley, FreeBSD development setup, HardenedBSD 2019 Status Report, DNSSEC, compiling RainbowCrack on OpenBSD, and more. Headlines FreeBSD 12.1 Some of the highlights: BearSSL has been imported to the base system. The clang, llvm, lld, lldb, compiler-rt utilities and libc++ have been updated to version 8.0.1. OpenSSL has been updated to version 1.1.1d. Several userland utility updates. For a complete list of new features and known problems, please see the online release notes and errata list, available at: https://www.FreeBSD.org/releases/12.1R/relnotes.html A History of UNIX before Berkeley: UNIX Evolution: 1975-1984. Nobody needs to be told that UNIX is popular today. In this article we will show you a little of where it was yesterday and over the past decade. And, without meaning in the least to minimise the incredible contributions of Ken Thompson and Dennis Ritchie, we will bring to light many of the others who worked on early versions, and try to show where some of the key ideas came from, and how they got into the UNIX of today. Our title says we are talking about UNIX evolution. Evolution means different things to different people. We use the term loosely, to describe the change over time among the many different UNIX variants in use both inside and outside Bell Labs. Ideas, code, and useful programs seem to have made their way back and forth - like mutant genes - among all the many UNIXes living in the phone company over the decade in question. Part One looks at some of the major components of the current UNIX system - the text formatting tools, the compilers and program development tools, and so on. Most of the work described in Part One took place at Research'', a part of Bell Laboratories (now AT&T Bell Laboratories, then as nowthe Labs''), and the ancestral home of UNIX. In planned (but not written) later parts, we would have looked at some of the myriad versions of UNIX - there are far more than one might suspect. This includes a look at Columbus and USG and at Berkeley Unix. You'll begin to get a glimpse inside the history of the major streams of development of the system during that time. News Roundup My FreeBSD Development Setup I do my FreeBSD development using git, tmux, vim and cscope. I keep a FreeBSD fork on my github, I have forked https://github.com/freebsd/freebsd to https://github.com/adventureloop/freebsd OPNsense 19.7.6 released As we are experiencing the Suricata community first hand in Amsterdam we thought to release this version a bit earlier than planned. Included is the latest Suricata 5.0.0 release in the development version. That means later this November we will releasing version 5 to the production version as we finish up tweaking the integration and maybe pick up 5.0.1 as it becomes available. LDAP TLS connectivity is now integrated into the system trust store, which ensures that all required root and intermediate certificates will be seen by the connection setup when they have been added to the authorities section. The same is true for trusting self-signed certificates. On top of this, IPsec now supports public key authentication as contributed by Pascal Mathis. HardenedBSD November 2019 Status Report. We at HardenedBSD have a lot of news to share. On 05 Nov 2019, Oliver Pinter resigned amicably from the project. All of us at HardenedBSD owe Oliver our gratitude and appreciation. This humble project, named by Oliver, was born out of his thesis work and the collaboration with Shawn Webb. Oliver created the HardenedBSD repo on GitHub in April 2013. The HardenedBSD Foundation was formed five years later to carry on this great work. DNSSEC enabled in default unbound(8) configuration. DNSSEC validation has been enabled in the default unbound.conf(5) in -current. The relevant commits were from Job Snijders (job@) How to Install Shopware with NGINX and Let's Encrypt on FreeBSD 12 Shopware is the next generation of open source e-commerce software. Based on bleeding edge technologies like Symfony 3, Doctrine2 and Zend Framework Shopware comes as the perfect platform for your next e-commerce project. This tutorial will walk you through the Shopware Community Edition (CE) installation on FreeBSD 12 system by using NGINX as a web server. Requirements Make sure your system meets the following minimum requirements: Linux-based operating system with NGINX or Apache 2.x (with mod_rewrite) web server installed. PHP 5.6.4 or higher with ctype, gd, curl, dom, hash, iconv, zip, json, mbstring, openssl, session, simplexml, xml, zlib, fileinfo, and pdo/mysql extensions. PHP 7.1 or above is strongly recommended. MySQL 5.5.0 or higher. Possibility to set up cron jobs. Minimum 4 GB available hard disk space. IonCube Loader version 5.0.0 or higher (optional). How to Compile RainbowCrack on OpenBSD Project RainbowCrack was originally Zhu Shuanglei's implementation, it's not clear to me if the project is still jus

Migrating drives and zpool between hosts, OpenBSD in 2019, Dragonfly’s new zlib and dhcpcd, Batch renaming images and resolution with awk, a rant on the X11 ICCCM selection system, hammer 2 emergency space mode, and more. Headlines Migrating drives and the zpool from one host to another. Today is the day. Today I move a zpool from an R710 into an R720. The goal: all services on that zpool start running on the new host. Fortunately, that zpool is dedicated to jails, more or less. I have done some planning about this, including moving a poudriere on the R710 into a jail. Now it is almost noon on Saturday, I am sitting in the basement (just outside the server room), and I’m typing this up. In this post: FreeBSD 12.0 Dell R710 (r710-01) Dell R720 (r720-01) drive caddies from eBay and now I know the difference between SATA and SATAu PLEASE READ THIS first: Migrating ZFS Storage Pools OpenBSD in 2019 I’ve used OpenBSD on and off since 2.1. More back then than in the last 10 years or so though, so I thought I’d try it again. What triggered this was me finding a silly bug in GNU cpio that has existed with a “FIXME” comment since at least 1994. I checked OpenBSD to see if it had a related bug, but as expected no it was just fine. I don’t quite remember why I stopped using OpenBSD for servers, but I do remember filesystem corruption on “unexpected power disconnections” (even with softdep turned on), which I’ve never really seen on Linux. That and that fewer things “just worked” than with Linux, which matters more when I installed more random things than I do now. I’ve become a lot more minimalist. Probably due to less spare time. Life is better when you don’t run things like PHP (not that OpenBSD doesn’t support PHP, just an example) or your own email server with various antispam tooling, and other things. This is all experience from running OpenBSD on a server. On my next laptop I intend to try running OpenBSD on the dektop, and will see if that more ad-hoc environment works well. E.g. will gnuradio work? Lack of other-OS VM support may be a problem. Verdict Ouch, that’s a long list of bad stuff. Still, I like it. I’ll continue to run it, and will make sure my stuff continues working on OpenBSD. And maybe in a year I’ll have a review of OpenBSD on a laptop. News Roundup New zlib, new dhcpcd zlib and dhcpcd are both updated in DragonFly… but my quick perusal of the commits makes it sound like bugfix only; no usage changes needed. DHCPCD Commit: http://lists.dragonflybsd.org/pipermail/commits/2019-October/719768.html ZLIB Commit: http://lists.dragonflybsd.org/pipermail/commits/2019-October/719772.html Batch renaming images, including image resolution, with awk The most recent item on my list of “Geeky things I did that made me feel pretty awesome” is an hour’s adventure that culminated in this code: $ file IMG* | awk 'BEGIN{a=0} {print substr($1, 1, length($1)-5),a++"_"substr($8,1, length($8)-1)}' | while read fn fr; do echo $(rename -v "s/$fn/img_$fr/g" *); done IMG_20170808_172653_425.jpg renamed as img_0_4032x3024.jpg IMG_20170808_173020_267.jpg renamed as img_1_3024x3506.jpg IMG_20170808_173130_616.jpg renamed as img_2_3024x3779.jpg IMG_20170808_173221_425.jpg renamed as img_3_3024x3780.jpg IMG_20170808_173417_059.jpg renamed as img_4_2956x2980.jpg IMG_20170808_173450_971.jpg renamed as img_5_3024x3024.jpg IMG_20170808_173536_034.jpg renamed as img_6_4032x3024.jpg IMG_20170808_173602_732.jpg renamed as img_7_1617x1617.jpg IMG_20170808_173645_339.jpg renamed as img_8_3024x3780.jpg IMG_20170909_170146_585.jpg renamed as img_9_3036x3036.jpg IMG_20170911_211522_543.jpg renamed as img_10_3036x3036.jpg IMG_20170913_071608_288.jpg renamed as img_11_2760x2760.jpg IMG_20170913_073205_522.jpg renamed as img_12_2738x2738.jpg // ... etc etc The last item on the aforementioned list is “TODO: come up with a shorter title for this list.” I hate the X11 ICCCM selection system, and you should too - A Rant d00d, that document is devilspawn. I've recently spent my nights in pain implementing the selection mechanism. WHY OH WHY OH WHY? why me? why did I choose to do this? and what sick evil twisted mind wrote this damn spec? I don't know why I'm working with it, I just wanted to make a useful program. I didn't know what I was getting myself in to. Nobody knows until they try it. And once you start, you're unable to stop. You can't stop, if you stop then you haven't completed it to spec. You can't fail on this, it's just a few pages of text, how can that be so hard? So what if they use Atoms for everything. So what if there's no explicit correlation between the target type of a SelectionNotify event and the type of the property it indicates? So what if the distinction is ambiguous? So what if the document is littered with such atrocities? It's not the spec's fault, the spec is authoritative. It's obviously YOUR (the implementor's) fault for misundersta

The earliest Unix code, how to replace fail2ban with blacklistd, OpenBSD crossed 400k commits, how to install Bolt CMS on FreeBSD, optimized hammer2, appeasing the OSI 7-layer burrito guys, and more. Headlines The Earliest Unix Code: An Anniversary Source Code Release What is it that runs the servers that hold our online world, be it the web or the cloud? What enables the mobile apps that are at the center of increasingly on-demand lives in the developed world and of mobile banking and messaging in the developing world? The answer is the operating system Unix and its many descendants: Linux, Android, BSD Unix, MacOS, iOS—the list goes on and on. Want to glimpse the Unix in your Mac? Open a Terminal window and enter “man roff” to view the Unix manual entry for an early text formatting program that lives within your operating system. 2019 marks the 50th anniversary of the start of Unix. In the summer of 1969, that same summer that saw humankind’s first steps on the surface of the Moon, computer scientists at the Bell Telephone Laboratories—most centrally Ken Thompson and Dennis Ritchie—began the construction of a new operating system, using a then-aging DEC PDP-7 computer at the labs. This man sent the first online message 50 years ago As many of you have heard in the past, the first online message ever sent between two computers was "lo", just over 50 years ago, on Oct. 29, 1969. It was supposed to say "log," but the computer sending the message — based at UCLA — crashed before the letter "g" was typed. A computer at Stanford 560 kilometres away was supposed to fill in the remaining characters "in," as in "log in." The CBC Radio show, “The Current” has a half-hour interview with the man who sent that message, Leonard Kleinrock, distinguished professor of computer science at UCLA "The idea of the network was you could sit at one computer, log on through the network to a remote computer and use its services there," 50 years later, the internet has become so ubiquitous that it has almost been rendered invisible. There's hardly an aspect in our daily lives that hasn't been touched and transformed by it. Q: Take us back to that day 50 years ago. Did you have the sense that this was going to be something you'd be talking about a half a century later? A: Well, yes and no. Four months before that message was sent, there was a press release that came out of UCLA in which it quotes me as describing what my vision for this network would become. Basically what it said is that this network would be always on, always available. Anybody with any device could get on at anytime from any location, and it would be invisible. Well, what I missed ... was that this is going to become a social network. People talking to people. Not computers talking to computers, but [the] human element. Q: Can you briefly explain what you were working on in that lab? Why were you trying to get computers to actually talk to one another? A: As an MIT graduate student, years before, I recognized I was surrounded by computers and I realized there was no effective [or efficient] way for them to communicate. I did my dissertation, my research, on establishing a mathematical theory of how these networks would work. But there was no such network existing. AT&T said it won't work and, even if it does, we want nothing to do with it. So I had to wait around for years until the Advanced Research Projects Agency within the Department of Defence decided they needed a network to connect together the computer scientists they were supervising and supporting. Q: For all the promise of the internet, it has also developed some dark sides that I'm guessing pioneers like yourselves never anticipated. A: We did not. I knew everybody on the internet at that time, and they were all well-behaved and they all believed in an open, shared free network. So we did not put in any security controls. When the first spam email occurred, we began to see the dark side emerge as this network reached nefarious people sitting in basements with a high-speed connection, reaching out to millions of people instantaneously, at no cost in time or money, anonymously until all sorts of unpleasant events occurred, which we called the dark side. But in those early days, I considered the network to be going through its teenage years. Hacking to spam, annoying kinds of effects. I thought that one day this network would mature and grow up. Well, in fact, it took a turn for the worse when nation states, organized crime and extremists came in and began to abuse the network in severe ways. Q: Is there any part of you that regrets giving birth to this? A: Absolutely not. The greater good is much more important. News Roundup How to use blacklistd(8) with NPF as a fail2ban replacement blacklistd(8) provides an API that can be used by network daemons to communicate with a packet filter via a daemon to enforce opening and c

Unix is 50, Hunting down Ken's PDP-7, OpenBSD and OPNSense have new releases, Clarification on what GhostBSD is, sshuttle - VPN over SSH, and more. Headlines Unix is 50 In the summer of 1969 computer scientists Ken Thompson and Dennis Ritchie created the first implementation of Unix with the goal of designing an elegant and economical operating system for a little-used PDP-7 minicomputer at Bell Labs. That modest project, however, would have a far-reaching legacy. Unix made large-scale networking of diverse computing systems — and the Internet — practical. The Unix team went on to develop the C language, which brought an unprecedented combination of efficiency and expressiveness to programming. Both made computing more "portable". Today, Linux, the most popular descendent of Unix, powers the vast majority of servers, and elements of Unix and Linux are found in most mobile devices. Meanwhile C++ remains one of the most widely used programming languages today. Unix may be a half-century old but its influence is only growing. Hunting down Ken's PDP-7: video footage found In my prior blog post, I traced Ken's scrounged PDP-7 to SN 34. In this post I'll show that we have actual video footage of that PDP-7 due to an old film from Bell Labs. this gives us almost a minute of footage of the PDP-7 Ken later used to create Unix. News Roundup OpenBSD 6.6 Released Announce: https://marc.info/?l=openbsd-tech&m=157132024225971&w=2 Upgrade Guide: https://openbsd.org/faq/upgrade66.html Changelog: https://openbsd.org/plus66.html OPNsense 19.7.5 released Hello friends and followers, Lots of plugin and ports updates this time with a few minor improvements in all core areas. Behind the scenes we are starting to migrate the base system to version 12.1 which is supposed to hit the next 20.1 release. Stay tuned for more infos in the next month or so. Here are the full patch notes: system: show all swap partitions in system information widget system: flatten services_get() in preparation for removal system: pin Syslog-ng version to specific package name system: fix LDAP/StartTLS with user import page system: fix a PHP warning on authentication server page system: replace most subprocess.call use interfaces: fix devd handling of carp devices (contributed by stumbaumr) firewall: improve firewall rules inline toggles firewall: only allow TCP flags on TCP protocol firewall: simplify help text for direction setting firewall: make protocol log summary case insensitive reporting: ignore malformed flow records captive portal: fix type mismatch for timeout read dhcp: add note for static lease limitation with lease registration (contributed by Northguy) ipsec: add margintime and rekeyfuzz options ipsec: clear $dpdline correctly if not set ui: fix tokenizer reorder on multiple saves plugins: os-acme-client 1.26[1] plugins: os-bind will reload bind on record change (contributed by blablup) plugins: os-etpro-telemetry minor subprocess.call replacement plugins: os-freeradius 1.9.4[2] plugins: os-frr 1.12[3] plugins: os-haproxy 2.19[4] plugins: os-mailtrail 1.2[5] plugins: os-postfix 1.11[6] plugins: os-rspamd 1.8[7] plugins: os-sunnyvalley LibreSSL support (contributed by Sunny Valley Networks) plugins: os-telegraf 1.7.6[8] plugins: os-theme-cicada 1.21 (contributed by Team Rebellion) plugins: os-theme-tukan 1.21 (contributed by Team Rebellion) plugins: os-tinc minor subprocess.call replacement plugins: os-tor 1.8 adds dormant mode disable option (contributed by Fabian Franz) plugins: os-virtualbox 1.0 (contributed by andrewhotlab) Dealing with the misunderstandings of what is GhostBSD Since the release of 19.09, I have seen a lot of misunderstandings on what is GhostBSD and the future of GhostBSD. GhostBSD is based on TrueOS with FreeBSD 12 STABLE with our twist to it. We are still continuing to use TrueOS for OpenRC, and the new package's system for the base system that is built from ports. GhostBSD is becoming a slow-moving rolling release base on the latest TrueOS with FreeBSD 12 STABLE. When FreeBSD 13 STABLE gets released, GhostBSD will be upgraded to TrueOS with FreeBSD 13 STABLE. Our official desktop is MATE, which means that the leading developer of GhostBSD does not officially support XFCE. Community releases are maintained by the community and for the community. GhostBSD project will provide help to build and to host the community release. If anyone wants to have a particular desktop supported, it is up to the community. Sure I will help where I can, answer questions and guide new community members that contribute to community release. There is some effort going on for Plasma5 desktop. If anyone is interested in helping with XFCE and Plasma5 or in creating another community release, you are well come to contribute. Also, Contribution to the GhostBSD base system, to ports and new ports, and in house software are welcome. We are mostly active on Telegram https://t.me/ghostbsd, but you can also reach

An interview with Trenton Schulz about his early days with FreeBSD, Robot OS, Qt, and more. Interview - Trenton Schulz - [email protected] Robot OS on FreeBSD BR: Welcome to the show. Can you tell us a little bit about yourself and how you got started with BSD? AJ: You were working for Trolltech (creators of Qt). Was FreeBSD used there and how? BR: Can you tell us more about the work you are doing with Robot OS on FreeBSD? AJ: Was EuroBSDcon your first BSD conference? How did you like it? BR: Do you have some tips or advice on how to get started with the BSDs? AJ: Is there anything else you’d like to tell us before we let you go? Beastie Bits FreeBSD Miniconf at linux.conf.au 2020 Call for Sessions Now Open Portland BSD Pizza Night: Oct 24th, 19:00 @ Rudy’s Gourmet Pizza NYC*BUG Install Fest: Nov 6th 18:45 @ Suspenders FOSDEM 2020 - BSD Devroom Call for Participation University of Cambridge looking for Research Assistants/Associates Send questions, comments, show ideas/topics, or stories you want mentioned on the show to [email protected] Your browser does not support the HTML5 video tag. Special Guest: Trenton Shulz.

Headlines FreeBSD and custom firmware on the Google Pixelbook FreeBSD and custom firmware on the Google Pixelbook Back in 2015, I jumped on the ThinkPad bandwagon by getting an X240 to run FreeBSD on. Unlike most people in the ThinkPad crowd, I actually liked the clickpad and didn\u2019t use the trackpoint much. But this summer I\u2019ve decided that it was time for something newer. I wanted something.. lighter and thinner (ha, turns out this is actually important, I got tired of carrying a T H I C C laptop - Apple was right all along); with a 3:2 display (why is Lenovo making these Serious Work\u2122 laptops 16:9 in the first place?? 16:9 is awful in below-13-inch sizes especially); with a HiDPI display (and ideally with a good size for exact 2x scaling instead of fractional); with USB-C ports; without a dGPU, especially without an NVIDIA GPU; assembled with screws and not glue (I don\u2019t necessarily need expansion and stuff in a laptop all that much, but being able to replace the battery without dealing with a glued chassis is good); supported by FreeBSD of course (\u201csome development required\u201d is okay but I\u2019m not going to write big drivers); how about something with open source firmware, that would be fun. I was considering a ThinkPad X1 Carbon from an old generation - the one from the same year as the X230 is corebootable, so that\u2019s fun. But going back in processor generations just doesn\u2019t feel great. I want something more efficient, not less! And then I discovered the Pixelbook. Other than the big huge large bezels around the screen, I liked everything about it. Thin aluminum design, a 3:2 HiDPI screen, rubber palm rests (why isn\u2019t every laptop ever doing that?!), the \u201cconvertibleness\u201d (flip the screen around to turn it into.. something rather big for a tablet, but it is useful actually), a Wacom touchscreen that supports a pen, mostly reasonable hardware (Intel Wi-Fi), and that famous coreboot support (Chromebooks\u2019 stock firmware is coreboot + depthcharge). So here it is, my new laptop, a Google Pixelbook. Conclusion Pixelbook, FreeBSD, coreboot, EDK2 good. Seriously, I have no big words to say, other than just recommending this laptop to FOSS enthusiasts :) Porting NetBSD to the AMD x86-64: a case study in OS portability Abstract NetBSD is known as a very portable operating system, currently running on 44 different architectures (12 different types of CPU). This paper takes a look at what has been done to make it portable, and how this has decreased the amount of effort needed to port NetBSD to a new architecture. The new AMD x86-64 architecture, of which the specifications were published at the end of 2000, with hardware to follow in 2002, is used as an example. Portability Supporting multiple platforms was a primary goal of the NetBSD project from the start. As NetBSD was ported to more and more platforms, the NetBSD kernel code was adapted to become more portable along the way. General Generally, code is shared between ports as much as possible. In NetBSD, it should always be considered if the code can be assumed to be useful on other architectures, present or future. If so, it is machine-independent and put it in an appropriate place in the source tree. When writing code that is intended to be machine-independent, and it contains conditional preprocessor statements depending on the architecture, then the code is likely wrong, or an extra abstraction layer is needed to get rid of these statements. Types Assumptions about the size of any type are not made. Assumptions made about type sizes on 32-bit platforms were a large problem when 64-bit platforms came around. Most of the problems of this kind had to be dealt with when NetBSD was ported to the DEC Alpha in 1994. A variation on this problem had to be dealt with with the UltraSPARC (sparc64) port in 1998, which is 64-bit, but big endian (vs. the little-endianness of the Alpha). When interacting with datastructures of a fixed size, such as on-disk metadata for filesystems, or datastructures directly interpreted by device hardware, explicitly sized types are used, such as uint32_t, int8_t, etc. Conclusions and future work The port of NetBSD to AMD's x86-64 architecture was done in six weeks, which confirms NetBSD's reputation as being a very portable operating system. One week was spent setting up the cross-toolchain and reading the x86-64 specifications, three weeks were spent writing the kernel code, one week was spent writing the userspace code, and one week testing and debugging it all. No problems were observed in any of the machine-independent parts of the kernel during test runs; all (simulated) device drivers, file systems, etc, worked without modification. News Roundup ZFS performance really does degrade as you approach quota limits Every so often (currently monthly), there is an "OpenZFS leadership meeting". What this really means is 'lead developers from the various ZFS

Causing ZFS corruption for fun, NetBSD Assembly Programming Tutorial, The IKEA Lack Rack for Servers, a new OmniOS Community Edition LTS has been published, List Block Devices on FreeBSD lsblk(8) Style, Project Trident 19.10 available, and more. Headlines Causing ZFS corruption for fun and profit Datto backs up data, a lot of it. At the time of writing Datto has over 500 PB of data stored on ZFS. This count includes both backup appliances that are sent to customer sites, as well as cloud storage servers that are used for secondary and tertiary backup of those appliances. At this scale drive swaps are a daily occurrence, and data corruption is inevitable. How we handle this corruption when it happens determines whether we truly lose data, or successfully restore from secondary backup. In this post we'll be showing you how at Datto we intentionally cause corruption in our testing environments, to ensure we're building software that can properly handle these scenarios. Causing Corruption Since this is a mirror setup, a naive solution to cause corruption would be to randomly dd the same sectors of both /dev/sdb and /dev/sdc. This works, but is equally likely to just overwrite random unused space, or take down the zpool entirely. What we really want is to corrupt a specific snapshot, or even a specific file in that snapshot, to simulate a more realistic minor corruption event. Luckily we have a tool called zdb that lets us view some low level information about datasets. Conclusion At the 500 PB scale, it's not a matter of if data corruption will happen but when. Intentionally causing corruption is one of the strategies we use to ensure we're building software that can handle these rare (but inevitable) events. To others out there using ZFS: I'm curious to hear how you've solved this problem. We did quite a bit of experimentation with zinject before going with this more brute force method. So I'd be especially interested if you've had luck simply simulating corruption with zinject. NetBSD Assembly Programming Tutorial A sparc64 version is also being prepared and will be added when done This post describes how to write a simple hello world program in pure assembly on NetBSD/amd64. We will not use (nor link against) libc, nor use gcc to compile it. I will be using GNU as (gas), and therefore the AT&T syntax instead of Intel. Why assembly? Why not? Because it's fun to program in assembly directly. Contrary to a popular belief assembly programs aren't always faster than what optimizing compilers produce. Nevertheless it's good to be able to read assembly, especially when debugging C programs Due to the nature of the guide, visit the site for the complete breakdown News Roundup The IKEA Lack Rack for Servers The LackRack First occurrence on eth0:2010 Winterlan, the LackRack is the ultimate, low-cost, high shininess solution for your modular datacenter-in-the-living-room. Featuring the LACK (side table) from Ikea, the LackRack is an easy-to-implement, exact-fit datacenter building block. It's a little known fact that we have seen Google engineers tinker with Lack tables since way back in 2009. The LackRack will certainly make its appearance again this summer at eth0:2010 Summer. Summary When temporarily not in use, multiple LackRacks can be stacked in a space-efficient way without disassembly, unlike competing 19" server racks. The LackRack was first seen on eth0:2010 Winterlan in the no-shoe Lounge area. Its low-cost and perfect fit are great for mounting up to 8 U of 19" hardware, such as switches (see below), or perhaps other 19" gear. It's very easy to assemble, and thanks to the design, they are stable enough to hold (for example) 19" switches and you can put your bottle of Club-Mate on top! Multi-shiny LackRack can also be painted to your specific preferences and the airflow is unprecedented! Howto You can find a howto on buying a LackRack on this page. This includes the proof that a 19" switch can indeed be placed in the LackRack in its natural habitat! OmniOS Community Edition r151030 LTS - Published at May 6, 2019 The OmniOS Community Edition Association is proud to announce the general availability of OmniOS - r151030. OmniOS is published according to a 6-month release cycle, r151030 LTS takes over from r151028, published in November 2018; and since it is a LTS release it also takes over from r151022. The r151030 LTS release will be supported for 3 Years. It is the first LTS release published by the OmniOS CE Association since taking over the reins from OmniTI in 2017. The next LTS release is scheduled for May 2021. The old stable r151026 release is now end-of-life. See the release schedule for further details. This is only a small selection of the new features, and bug fixes in the new release; review the release notes for full details. If you upgrade from r22 and want to see all new features added since then, make sure to also

DragonFlyBSD vs. FreeBSD vs. Linux benchmark on Ryzen 7, JFK Presidential Library chooses TrueNAS for digital archives, FreeBSD 12.1-beta is available, cool but obscure X11 tools, vBSDcon trip report, Project Trident 12-U7 is available, a couple new Unix artifacts, and more. Headlines DragonFlyBSD 5.6 vs. FreeBSD 12 vs. Linux - Ryzen 7 3700X For those wondering how well FreeBSD and DragonFlyBSD are handling AMD's new Ryzen 3000 series desktop processors, here are some benchmarks on a Ryzen 7 3700X with MSI MEG X570 GODLIKE where both of these popular BSD operating systems were working out-of-the-box. For some fun mid-week benchmarking, here are those results of FreeBSD 12.0 and DragonFlyBSD 5.6.2 up against openSUSE Tumbleweed and Ubuntu 19.04. Back in July I looked at FreeBSD 12 on the Ryzen 9 3900X but at that time at least DragonFlyBSD had troubles booting on that system. When trying out the Ryzen 7 3700X + MSI GODLIKE X570 motherboard on the latest BIOS, everything "just worked" without any compatibility issues for either of these BSDs. We've been eager to see how well DragonFlyBSD is performing on these new AMD Zen 2 CPUs with DragonFlyBSD lead developer Matthew Dillon having publicly expressed being impressed by the new AMD Ryzen 3000 series CPUs. For comparison to those BSDs, Ubuntu 19.04 and openSUSE Tumbleweed were tested on the same hardware in their out-of-the-box configurations. While Clear Linux is normally the fastest, on this system Clear's power management defaults had caused issues in being unable to detect the Samsung 970 EVO Plus NVMe SSD used for testing and so we left it out this round. All of the hardware was the same throughout testing as were the BIOS settings and running the Ryzen 7 3700X at stock speeds. (Any differences in the reported hardware for the system table just come down to differences in what is exposed by each OS for reporting.) All of the BSD/Linux benchmarks on this eight core / sixteen thread processor were run via the Phoronix Test Suite. In the case of FreeBSD 12.0, we benchmarked both with its default LLVM Clang 6.0 compiler as well as with GCC 9.1 so that it would match the GCC compiler being the default on the other operating systems under test. JFK Presidential Library Chooses iXsystems TrueNAS to Preserve Precious Digital Archives iXsystems is honored to have the TrueNAS® M-Series unified storage selected to store, serve, and protect the entire digital archive for the John F. Kennedy Library Foundation. This is in support of the collection at the John F. Kennedy Presidential Library and Museum (JFK Library). Over the next several years, the Foundation hopes to grow the digital collection from hundreds of terabytes today to cover much more of the Archives at the Kennedy Library. Overall there is a total of 25 million documents, audio recordings, photos, and videos once the project is complete. Having first deployed the TrueNAS M50-HA earlier in 2019, the JFK Library has now completed the migration of its existing digital collection and is now in the process of digitizing much of the rest of its vast collection. Not only is the catalog of material vast, it is also diverse, with files being copied to the storage system from a variety of sources in numerous file types. To achieve this ambitious goal, the library required a high-end NAS system capable of sharing with a variety of systems throughout the digitization process. The digital archive will be served from the TrueNAS M50 and made available to both in-person and online visitors. With precious material and information comes robust demands. The highly-available TrueNAS M-Series has multiple layers of protection to help keep data safe, including data scrubs, checksums, unlimited snapshots, replication, and more. TrueNAS is also inherently scalable with data shares only limited by the number of drives connected to the pool. Perfect for archival storage, the deployed TrueNAS M50 will grow with the library’s content, easily expanding its storage capacity over time as needed. Supporting a variety of protocols, multi-petabyte scalability in a single share, and anytime, uninterrupted capacity expansion, the TrueNAS M-Series ticked all the right boxes. Youtube Video News Roundup FreeBSD 12.1-beta available FreeBSD 12.0 is already approaching one year old while FreeBSD 12.1 is now on the way as the next installment with various bug/security fixes and other alterations to this BSD operating system. FreeBSD 12.1 has many security/bug fixes throughout, no longer enables "-Werror" by default as a compiler flag (Update: This change is just for the GCC 4.2 compiler), has imported BearSSL into the FreeBSD base system as a lightweight TLS/SSL implementation, bzip2recover has been added, and a variety of mostly lower-level changes. More details can be found via the in-progress release notes. For those with time to test this weekend, FreeBSD 12.1 Beta 1 is available for all prominent architectures

Setting up buildbot in FreeBSD jails, Set up a mail server with OpenSMTPD, Dovecot and Rspamd, OpenBSD amateur packet radio with HamBSD, DragonFlyBSD's HAMMER2 gets fsck, return of startx for users. Headlines EuroBSDcon 2019 Recap We’re back from EuroBSDcon in Lillehammer, Norway. It was a great conference with 212 people attending. 2 days of tutorials, parallel to the FreeBSD Devsummit, followed by two days of talks. Some speakers uploaded their slides to papers.freebsd.org already with more to come. The social event was also interesting. We visited an open air museum with building preserved from different time periods. In the older section they had a collection of farm buildings, a church originally built in the 1200s and relocated to the museum, and a school house. In the more modern area, they had houses from 1915, and each decade from 1930 to 1990, plus a “house of the future” as imagined in 2001. Many had open doors to allow you to tour the inside, and some were even “inhabited”. The latter fact gave a much more interactive experience and we could learn additional things about the history of that particular house. The town at the end included a general store, a post office, and more. Then, we all had a nice dinner together in the museum’s restaurant. The opening keynote by Patricia Aas was very good. Her talk on embedded ethics, from her perspective as someone trying to defend the sanctity of Norwegian elections, and a former developer for the Opera web browser, provided a great deal of insight into the issues. Her points about how the tech community has unleashed a very complex digital work upon people with barely any technical literacy were well taken. Her stories of trying to explain the problems with involving computers in the election process to journalists and politicians struck a chord with many of us, who have had to deal with legislation written by those who do not truly understand the issues with technology. Setting up buildbot in FreeBSD jails In this article, I would like to present a tutorial to set up buildbot, a continuous integration (CI) software (like Jenkins, drone, etc.), making use of FreeBSD’s containerization mechanism "jails". We will cover terminology, rationale for using both buildbot and jails together, and installation steps. At the end, you will have a working buildbot instance using its sample build configuration, ready to play around with your own CI plans (or even CD, it’s very flexible!). Some hints for production-grade installations are given, but the tutorial steps are meant for a test environment (namely a virtual machine). Buildbot’s configuration and detailed concepts are not in scope here. Setting up a mail server with OpenSMTPD, Dovecot and Rspamd Self-hosting and encouraging smaller providers is for the greater good First of all, I was not clear enough about the political consequences of centralizing mail services at Big Mailer Corps. It doesn’t make sense for Random Joe, sharing kitten pictures with his family and friends, to build a personal mail infrastructure when multiple Big Mailer Corps offer “for free” an amazing quality of service. They provide him with an e-mail address that is immediately available and which will generally work reliably. It really doesn’t make sense for Random Joe not to go there, and particularly if even techies go there without hesitation, proving it is a sound choice. There is nothing wrong with Random Joes using a service that works. What is terribly wrong though is the centralization of a communication protocol in the hands of a few commercial companies, EVERY SINGLE ONE OF THEM coming from the same country (currently led by a lunatic who abuses power and probably suffers from NPD), EVERY SINGLE ONE OF THEM having been in the news and/or in a court for random/assorted “unpleasant” behaviors (privacy abuses, eavesdropping, monopoly abuse, sexual or professional harassment, you just name it…), and EVERY SINGLE ONE OF THEM growing user bases that far exceeds the total population of multiple countries combined. News Roundup The HamBSD project aims to bring amateur packet radio to OpenBSD The HamBSD project aims to bring amateur packet radio to OpenBSD, including support for TCP/IP over AX.25 and APRS tracking/digipeating in the base system. HamBSD will not provide a full AX.25 stack but instead only implement support for UI frames. There will be a focus on simplicity, security and readable code. The amateur radio community needs a reliable platform for packet radio for use in both leisure and emergency scenarios. It should be expected that the system is stable and resilient (but as yet it is neither). DragonFlyBSD's HAMMER2 Gets Basic FSCK Support HAMMER2 is Copy on Write, meaning changes are made to copies of existing data. This means operations are generally atomic and can survive a power outage, etc. (You should read up on it!) However, there’s now a fsck command, useful if you want a report of data validity rat

NetBSD LLVM sanitizers and GDB regression test suite, Ada—The Language of Cost Savings, Homura - a Windows Games Launcher for FreeBSD, FreeBSD core team appoints a WG to explore transition to Git, OpenBSD 6.6 Beta tagged, Project Trident 12-U5 update now available, and more. Headlines LLVM santizers and GDB regression test suite. As NetBSD-9 is branched, I have been asked to finish the LLVM sanitizer integration. This work is now accomplished and with MKLLVM=yes build option (by default off), the distribution will be populated with LLVM files for ASan, TSan, MSan, UBSan, libFuzzer, SafeStack and XRay. I have also transplanted basesystem GDB patched to my GDB repository and managed to run the GDB regression test-suite. NetBSD distribution changes I have enhanced and imported my local MKSANITIZER code that makes whole distribution sanitization possible. Few real bugs were fixed and a number of patches were newly written to reflect the current NetBSD sources state. I have also merged another chunk of the fruits of the GSoC-2018 project with fuzzing the userland (by plusun@). The following changes were committed to the sources: ab7de18d0283 Cherry-pick upstream compiler-rt patches for LLVM sanitizers 966c62a34e30 Add LLVM sanitizers in the MKLLVM=yes build 8367b667adb9 telnetd: Stop defining the same variables concurrently in bss and data fe72740f64bf fsck: Stop defining the same variable concurrently in bss and data 40e89e890d66 Fix build of t_ubsan/t_ubsanxx under MKSANITIZER b71326fd7b67 Avoid symbol clashes in tests/usr.bin/id under MKSANITIZER c581f2e39fa5 Avoid symbol clashes in fs/nfs/nfsservice under MKSANITIZER 030a4686a3c6 Avoid symbol clashes in bin/df under MKSANITIZER fd9679f6e8b1 Avoid symbol clashes in usr.sbin/ypserv/ypserv under MKSANITIZER 5df2d7939ce3 Stop defining _rpcsvcdirty in bss and data 5fafbe8b8f64 Add missing extern declaration of ib_mach_emips in installboot d134584be69a Add SANITIZER_RENAME_CLASSES in bsd.prog.mk 2d00d9b08eae Adapt tests/kernel/t_subr_prf for MKSANITIZER ce54363fe452 Ship with sanitizer/lsan_interface.h for GCC 7 7bd5ee95e9a0 Ship with sanitizer/lsan_interface.h for LLVM 7 d8671fba7a78 Set NODEBUG for LLVM sanitizers 242cd44890a2 Add PAXCTL_FLAG rules for MKSANITIZER 5e80ab99d9ce Avoid symbol clashes in test/rump/modautoload/t_modautoload with sanitizers e7ce7ecd9c2a sysctl: Add indirection of symbols to remove clash with sanitizers 231aea846aba traceroute: Add indirection of symbol to remove clash with sanitizers 8d85053f487c sockstat: Add indirection of symbols to remove clash with sanitizers 81b333ab151a netstat: Add indirection of symbols to remove clash with sanitizers a472baefefe8 Correct the memset(3)'s third argument in i386 biosdisk.c 7e4e92115bc3 Add ATF c and c++ tests for TSan, MSan, libFuzzer 921ddc9bc97c Set NOSANITIZER in i386 ramdisk image 64361771c78d Enhance MKSANITIZER support 3b5608f80a2b Define target_not_supported_body() in TSan, MSan and libFuzzer tests c27f4619d513 Avoids signedness bit shift in db_get_value() 680c5b3cc24f Fix LLVM sanitizer build by GCC (HAVE_LLVM=no) 4ecfbbba2f2a Rework the LLVM compiler_rt build rules 748813da5547 Correct the build rules of LLVM sanitizers 20e223156dee Enhance the support of LLVM sanitizers 0bb38eb2f20d Register syms.extra in LLVM sanitizer .syms files Almost all of the mentioned commits were backported to NetBSD-9 and will land 9.0. Homura - a Windows Games Launcher for FreeBSD Inspired by lutris (a Linux gaming platform), we would like to provide a game launcher to play windows games on FreeBSD. Makes it easier to run games on FreeBSD, by providing the tweaks and dependencies for you Dependencies curl bash p7zip zenity webfonts alsa-utils (Optional) winetricks vulkan-tools mesa-demos i386-wine-devel on amd64 or wine-devel on i386 News Roundup Ada—The Language of Cost Savings? Many myths surround the Ada programming language, but it continues to be used and evolve at the same time. And while the increased adoption of Ada and SPARK, its provable subset, is slow, it’s noticeable. Ada already addresses more of the features found in found in heavily used embedded languages like C+ and C#. It also tackles problems addressed by upcoming languages like Rust. Chris concludes, “Development technologies have a profound impact on one of the largest and most variable costs associated with embedded-system engineering—labor. At a time when on-time system deployment can not only impact customer satisfaction, but access to services revenue streams, engineering team efficiency is at a premium. Our research showed that programming language choices can have significant influence in this area, leading to shorter projects, better schedules and, ultimately, lower development costs. While a variety of factors can influence and dictate language choice, our research showed that Ada’s evolution has made it an increasingly compelling option for engineering organizations, providing both technically and financially sound solutio

vBSDcon 2019 recap, Unix at 50, OpenBSD on fan-less Tuxedo InfinityBook, humungus - an hg server, how to configure a network dump in FreeBSD, and more. Headlines vBSDcon Recap Allan and Benedict attended vBSDcon 2019, which ended last week. It was held again at the Hyatt Regency Reston and the main conference was organized by Dan Langille of BSDCan fame.The two day conference was preceded by a one day FreeBSD hackathon, where FreeBSD developers had the chance to work on patches and PRs. In the evening, a reception was held to welcome attendees and give them a chance to chat and get to know each other over food and drinks. The first day of the conference was opened with a Keynote by Paul Vixie about DNS over HTTPS (DoH). He explained how we got to the current state and what challenges (technical and social) this entails. If you missed this talk and are dying to see it, it will also be presented at EuroBSDCon next week John Baldwin followed up by giving an overview of the work on “In-Kernel TLS Framing and Encryption for FreeBSD” abstract and the recent commit we covered in episode 313. Meanwhile, Brian Callahan was giving a separate session in another room about “Learning to (Open)BSD through its porting system: an attendee-driven educational session” where people had the chance to learn about how to create ports for the BSDs. David Fullard’s talk about “Transitioning from FreeNAS to FreeBSD” was his first talk at a BSD conference and described how he built his own home NAS setup trying to replicate FreeNAS’ functionality on FreeBSD, and why he transitioned from using an appliance to using vanilla FreeBSD. Shawn Webb followed with his overview talk about the “State of the Hardened Union”. Benedict’s talk about “Replacing an Oracle Server with FreeBSD, OpenZFS, and PostgreSQL” was well received as people are interested in how we liberated ourselves from the clutches of Oracle without compromising functionality. Entertaining and educational at the same time, Michael W. Lucas talk about “Twenty Years in Jail: FreeBSD Jails, Then and Now” closed the first day. Lucas also had a table in the hallway with his various tech and non-tech books for sale. People formed small groups and went into town for dinner. Some returned later that night to some work in the hacker lounge or talk amongst fellow BSD enthusiasts. Colin Percival was the keynote speaker for the second day and had an in-depth look at “23 years of software side channel attacks”. Allan reprised his “ELI5: ZFS Caching” talk explaining how the ZFS adaptive replacement cache (ARC) work and how it can be tuned for various workloads. “By the numbers: ZFS Performance Results from Six Operating Systems and Their Derivatives” by Michael Dexter followed with his approach to benchmarking OpenZFS on various platforms. Conor Beh was also a new speaker to vBSDcon. His talk was about “FreeBSD at Work: Building Network and Storage Infrastructure with pfSense and FreeNAS”. Two OpenBSD talks closed the talk session: Kurt Mosiejczuk with “Care and Feeding of OpenBSD Porters” and Aaron Poffenberger with “Road Warrior Disaster Recovery: Secure, Synchronized, and Backed-up”. A dinner and reception was enjoyed by the attendees and gave more time to discuss the talks given and other things until late at night. We want to thank the vBSDcon organizers and especially Dan Langille for running such a great conference. We are grateful to Verisign as the main sponsor and The FreeBSD Foundation for sponsoring the tote bags. Thanks to all the speakers and attendees! humungus - an hg server Features View changes, files, changesets, etc. Some syntax highlighting. Read only. Serves multiple repositories. Allows cloning via the obvious URL. Supports go get. Serves files for downloads. Online documentation via mandoc. Terminal based admin interface. News Roundup OpenBSD on fan-less Tuxedo InfinityBook 14″ v2. The InfinityBook 14” v2 is a fanless 14” notebook. It is an excellent choice for running OpenBSD - but order it with the supported wireless card (see below.). I’ve set it up in a dual-boot configuration so that I can switch between Linux and OpenBSD - mainly to spot differences in the drivers. TUXEDO allows a variety of configurations through their webshop. The dual boot setup with grub2 and EFI boot will be covered in a separate blogpost. My tests were done with OpenBSD-current - which is as of writing flagged as 6.6-beta. See Article for breakdown of CPU, Wireless, Video, Webcam, Audio, ACPI, Battery, Touchpad, and MicroSD Card Reader Unix at 50: How the OS that powered smartphones started from failure Maybe its pervasiveness has long obscured its origins. But Unix, the operating system that in one derivative or another powers nearly all smartphones sold worldwide, was born 50 years ago from the failure of an ambitious project that involved titans like Bell Labs, GE, and MIT. Largely the brainchild of a few programmers at Bell Labs, the unlikely story of Unix begins with a meeting on t

Unix virtual memory when you have no swap space, Dsynth details on Dragonfly, Instant Workstation on FreeBSD, new servers new tech, Experimenting with streaming setups on NetBSD, NetBSD’s progress towards Steam support thanks to GSoC, and more. Headlines What has to happen with Unix virtual memory when you have no swap space Recently, Artem S. Tashkinov wrote on the Linux kernel mailing list about a Linux problem under memory pressure (via, and threaded here). The specific reproduction instructions involved having low RAM, turning off swap space, and then putting the system under load, and when that happened (emphasis mine): Once you hit a situation when opening a new tab requires more RAM than is currently available, the system will stall hard. You will barely be able to move the mouse pointer. Your disk LED will be flashing incessantly (I'm not entirely sure why). [...] I'm afraid I have bad news for the people snickering at Linux here; if you're running without swap space, you can probably get any Unix to behave this way under memory pressure. If you can't on your particular Unix, I'd actually say that your Unix is probably not letting you get full use out of your RAM. To simplify a bit, we can divide pages of user memory up into anonymous pages and file-backed pages. File-backed pages are what they sound like; they come from some specific file on the filesystem that they can be written out to (if they're dirty) or read back in from. Anonymous pages are not backed by a file, so the only place they can be written out to and read back in from is swap space. Anonymous pages mostly come from dynamic memory allocations and from modifying the program's global variables and data; file backed pages come mostly from mapping files into memory with mmap() and also, crucially, from the code and read-only data of the program. See link for the rest of the article Dsynth details on Dragonfly First, history: DragonFly has had binaries of dports available for download for quite some time. These were originally built using poudriere, and then using the synth tool put together by John Marino. Synth worked both to build all software in dports, and as a way to test DragonFly’s SMP capability under extreme load. Matthew Dillon is working on a new version, called dsynth. It is available now but not yet part of the build. He’s been working quickly on it and there’s plenty more commits than what I have linked here. It’s already led to finding more high-load fixes. dsynth DSynth is basically synth written in C, from scratch. It is designed to give us a bulk builder in base and be friendly to porting and jails down the line (for now its uses chroot's). The original synth was written by John R. Marino and its basic flow was used in writing this program, but as it was written in ada no code was directly copied. The intent is to make dsynth compatible with synth's configuration files and directory structure. This is a work in progress and not yet ready for prime-time. Pushing so we can get some more eyeballs. Most of the directives do not yet work (everything, and build works, and 'cleanup' can be used to clean up any dangling mounts). dsynth code News Roundup Instant Workstation Some considerable time ago I wrote up instructions on how to set up a FreeBSD machine with the latest KDE Plasma Desktop. Those instructions, while fairly short (set up X, install the KDE meta-port, .. and that’s it) are a bit fiddly. So – prompted slightly by a Twitter exchange recently – I’ve started a mini-sub-project to script the installation of a desktop environment and the bits needed to support it. To give it at least a modicum of UI, dialog(1) is used to ask for an environment to install and a display manager. The tricky bits – pointed out to me after I started – are hardware support, although a best-effort is better than having nothing, I think. In any case, in a VBox host it’s now down to running a single script and picking Plasma and SDDM to get a usable system for me. Other combinations have not been tested, nor has system-hardware-setup. I’ll probably maintain it for a while and if I have time and energy it’ll be tried with nVidia (those work quite well on FreeBSD) and AMD (not so much, in my experience) graphics cards when I shuffle some machines around. Here is the script in my GitHub repository with notes-for-myself. New Servers, new Tech Following up on an earlier post, the new servers for DragonFly are in place. The old 40-core machine used for bulk build, monster, is being retired. The power efficiency of the new machines is startling. Incidentally, this is where donations go – infrastructure. New servers in the colo, monster is being retired We have three new servers in the colo now that will be taking most/all bulk package building duties from monster and the two blades (muscles and pkgbox64) that previously did the work. Monster will be retired. The new servers are a dual-socket Xeon (sting)

OpenBSD on 7th gen Thinkpad X1 Carbon, how to install FreeBSD on a MacBook, Kernel portion of in-kernel TLS (KTLS), Boot Environments on DragonflyBSD, Project Trident Updates, vBSDcon schedule, and more. Headlines OpenBSD on the Thinkpad X1 Carbon 7th Gen Another year, another ThinkPad X1 Carbon, this time with a Dolby Atmos sound system and a smaller battery. The seventh generation X1 Carbon isn't much different than the fifth and sixth generations. I opted for the non-vPro Core i5-8265U, 16Gb of RAM, a 512Gb NVMe SSD, and a matte non-touch WQHD display at ~300 nits. A brighter 500-nit 4k display is available, though early reports indicated it severely impacts battery life. Gone are the microSD card slot on the back and 1mm of overall thickness (from 15.95mm to 14.95mm), but also 6Whr of battery (down to 51Whr) and a little bit of travel in the keyboard and TrackPoint buttons. I still very much like the feel of both of them, so kudos to Lenovo for not going too far down the Apple route of sacrificing performance and usability just for a thinner profile. On my fifth generation X1 Carbon, I used a vinyl plotter to cut out stickers to cover the webcam, "X1 Carbon" branding from the bottom of the display, the power button LED, and the "ThinkPad" branding from the lower part of the keyboard deck. See link for the rest of the article How To Install FreeBSD On A MacBook 1,1 or 2,1 FreeBSD Setup For MacBook 1,1 and 2,1 FreeBSD with some additional setup can be installed on a MacBook 1,1 or 2,1. This article covers how to do so with FreeBSD 10-12. Installing FreeBSD can be installed as the only OS on your MacBook if desired. What you should have is: A Mac OS X 10.4.6-10.7.5 installer. Unofficial versions modified for these MacBooks such as 10.8 also work. A blank CD or DVD to burn the FreeBSD image to. Discs simply work best with these older MacBooks. An ISO file of FreeBSD for x86. The AMD64 ISO does not boot due to the 32 bit EFI of these MacBooks. Burn the ISO file to the blank CD or DVD. Once done, make sure it's in your MacBook and then power off the MacBook. Turn it on, and hold down the c key until the FreeBSD disc boots. See link for the rest of the guide News Roundup Patch for review: Kernel portion of in-kernel TLS (KTLS) One of the projects I have been working on for the past several months in conjunction with several other folks is upstreaming work from Netflix to handle some aspects of Transport Layer Security (TLS) in the kernel. In particular, this lets a web server use sendfile() to send static content on HTTPS connections. There is a lot more detail in the review itself, so I will spare pasting a big wall of text here. However, I have posted the patch to add the kernel-side of KTLS for review at the URL below. KTLS also requires other patches to OpenSSL and nginx, but this review is only for the kernel bits. Patches and reviews for the other bits will follow later. https://reviews.freebsd.org/D21277 DragonFly Boot Enviroments This is a tool inspired by the beadm utility for FreeBSD/Illumos systems that creates and manages ZFS boot environments. This utility in contrast is written from the ground up in C, this should provide better performance, integration, and extensibility than the POSIX sh and awk script it was inspired by. During the time this project has been worked on, beadm has been superseded by bectl on FreeBSD. After hammering out some of the outstanding internal logic issues, I might look at providing a similar interface to the command as bectl. See link for the rest of the details Project Trident Updates 19.08 Available This is a general package update to the CURRENT release repository based upon TrueOS 19.08. Legacy boot ISO functional again This update includes the FreeBSD fixes for the “vesa” graphics driver for legacy-boot systems. The system can once again be installed on legacy-boot systems. PACKAGE CHANGES FROM 19.07-U1 New Packages: 154 Deleted Packages: 394 Updated Packages: 4926 12-U3 Available This is the third general package update to the STABLE release repository based upon TrueOS 12-Stable. PACKAGE CHANGES FROM STABLE 12-U2 New Packages: 105 Deleted Packages: 386 Updated Packages: 1046 vBSDcon vBSDcon 2019 will return to the Hyatt Regency in Reston, VA on September 5-7 2019. *** Beastie Bits The next NYCBUG meeting will be Sept 4 @ 18:45 Feedback/Questions Tom - Questions Michael - dfbeadm Bostjan - Questions Send questions, comments, show ideas/topics, or stories you want mentioned on the show to [email protected] Your browser does not support the HTML5 video tag.

The UNIX Philosophy in 2019, why use package managers, touchpad interrupted, Porting wine to amd64 on NetBSD second evaluation report, Enhancing Syzkaller Support for NetBSD, all about the Pinebook Pro, killing a process and all of its descendants, fast software the best software, and more. Headlines The UNIX Philosophy in 2019 Today, Linux and open source rules the world, and the UNIX philosophy is widely considered compulsory. Organizations are striving to build small, focused applications that work collaboratively in a cloud and microservices environment. We rely on the network, as well as HTTP (text) APIs for storing and referencing data. Moreover, nearly all configuration is stored and communicated using text (e.g. YAML, JSON or XML). And while the UNIX philosophy has changed dramatically over the past 5 decades, it hasn’t strayed too far from Ken Thompson’s original definition in 1973: We write programs that do one thing and do it well We write programs to work together And we write programs that handle text streams, because that is a universal interface Why Use Package Managers? Valuable research is often hindered or outright prevented by the inability to install software. This need not be the case. Since I began supporting research computing in 1999, I’ve frequently seen researchers struggle for days or weeks trying to install a single open source application. In most cases, they ultimately failed. In many cases, they could have easily installed the software in seconds with one simple command, using a package manager such as Debian packages, FreeBSD ports, MacPorts, or Pkgsrc, just to name a few. Developer websites often contain poorly written instructions for doing “caveman installs”; manually downloading, unpacking, patching, and building the software. The same laborious process must often be followed for other software packages on which it depends, which can sometimes number in the dozens. Many researchers are simply unaware that there are easier ways to install the software they need. Caveman installs are a colossal waste of man-hours. If 1000 people around the globe spend an average of 20 hours each trying to install the same program that could have been installed with a package manager (this is not uncommon), then 20,000 man-hours have been lost that could have gone toward science. How many important discoveries are delayed by this? The elite research institutions have ample funding and dozens of IT staff dedicated to research computing. They can churn out publications even if their operation is inefficient. Most institutions, however, have few or no IT staff dedicated to research, and cannot afford to squander precious man-hours on temporary, one-off software installs. The wise approach for those of us in that situation is to collaborate on making software deployment easier for everyone. If we do so, then even the smallest research groups can leverage that work to be more productive and make more frequent contributions to science. Fortunately, the vast majority of open source software installs can be made trivial for anyone to do for themselves. Modern package managers perform all the same steps as a caveman install, but automatically. Package managers also install dependencies for us automatically. News Roundup Touchpad, Interrupted For two years I've been driving myself crazy trying to figure out the source of a driver problem on OpenBSD: interrupts never arrived for certain touchpad devices. A couple weeks ago, I put out a public plea asking for help in case any non-OpenBSD developers recognized the problem, but while debugging an unrelated issue over the weekend, I finally solved it. It's been a long journey and it's a technical tale, but here it is. Porting wine to amd64 on NetBSD, second evaluation report Summary Presently, Wine on amd64 is in test phase. It seems to work fine with caveats like LD_LIBRARY_PATH which has to be set as 32-bit Xorg libs don't have ${PREFIX}/emul/netbsd32/lib in its rpath section. The latter is due to us extracting 32-bit libs from tarballs in lieu of building 32-bit Xorg on amd64. As previously stated, pkgsrc doesn't search for pkgconfig files in ${PREFIX}/emul/netbsd32/lib which might have inadvertent effects that I am unaware of as of now. I shall be working on these issues during the final coding period. I would like to thank @leot, @maya and @christos for saving me from shooting myself in the foot many a time. I, admittedly, have had times when multiple approaches, which all seemed right at that time, perplexed me. I believe those are times when having a mentor counts, and I have been lucky enough to have really good ones. Once again, thanks to Google for this wonderful opportunity. Enhancing Syzkaller Support for NetBSD, Part 2 As a part of Google Summer of Code’19, I am working on improving the support for Syzkaller kernel fuzzer. Syzkaller is an unsupervised coverage-guided kernel fuzzer, that supports a variety of operating syste

NetBSD 9.0 release process has started, xargs, a tale of two spellcheckers, Adapting TriforceAFL for NetBSD, Exploiting a no-name freebsd kernel vulnerability, and more. Headlines NetBSD 9.0 release process has started If you have been following source-changes, you may have noticed the creation of the netbsd-9 branch! It has some really exciting items that we worked on: New AArch64 architecture support: Symmetric and asymmetrical multiprocessing support (aka big.LITTLE) Support for running 32-bit binaries UEFI and ACPI support Support for SBSA/SBBR (server-class) hardware. The FDT-ization of many ARM boards: the 32-bit GENERIC kernel lists 129 different DTS configurations the 64-bit GENERIC64 kernel lists 74 different DTS configurations All supported by a single kernel, without requiring per-board configuration. Graphics driver update, matching Linux 4.4, adding support for up to Kaby Lake based Intel graphics devices. ZFS has been updated to a modern version and seen many bugfixes. New hardware-accelerated virtualization via NVMM. NPF performance improvements and bug fixes. A new lookup algorithm, thmap, is now the default. NVMe performance improvements Optional kernel ASLR support, and partial kernel ASLR for the default configuration. Kernel sanitizers: KLEAK, detecting memory leaks KASAN, detecting memory overruns KUBSAN, detecting undefined behaviour These have been used together with continuous fuzzing via the syzkaller project to find many bugs that were fixed. The removal of outdated networking components such as ISDN and all of its drivers The installer is now capable of performing GPT UEFI installations. Dramatically improved support for userland sanitizers, as well as the option to build all of NetBSD's userland using them for bug-finding. Update to graphics userland: Mesa was updated to 18.3.4, and llvmpipe is now available for several architectures, providing 3D graphics even in the absence of a supported GPU. We try to test NetBSD as best as we can, but your testing can help NetBSD 9.0 a great release. Please test it and let us know of any bugs you find. Binaries are available at https://nycdn.netbsd.org/pub/NetBSD-daily/netbsd-9/latest/ xargs wtf xargs is probably one of the more difficult to understand of the unix command arsenal and of course that just means it’s one of the most useful too. I discovered a handy trick that I thought was worth a share. Please note there are probably other (better) ways to do this but I did my stackoverflow research and found nothing better. xargs — at least how I’ve most utilized it — is handy for taking some number of lines as input and doing some work per line. It’s hard to be more specific than that as it does so much else. It literally took me an hour of piecing together random man pages + tips from 11 year olds on stack overflow, but eventually I produced this gem: This is an example of how to find files matching a certain pattern and rename each of them. It sounds so trivial (and it is) but it demonstrates some cool tricks in an easy concept. News Roundup PkgSrc: A Tale of Two Spellcheckers This is a transcript of the talk I gave at pkgsrcCon 2019 in Cambridge, UK. It is about spellcheckers, but there are much more general software engineering lessons that we can learn from this case study. The reason I got into this subject at all was my paternal leave last year, when I finally had some more time to spend working on pkgsrc. It was a tiny item in the enormous TODO file at the top of the source tree (“update enchant to version 2.2”) that made me go into this rabbit hole. Adapting TriforceAFL for NetBSD, Part 2 I have been working on adapting TriforceAFL for NetBSD kernel syscall fuzzing. This blog post summarizes the work done until the second evaluation. For work done during the first coding period, check out this post. Summary > So far, the TriforceNetBSDSyscallFuzzer has been made available in the form of a pkgsrc package with the ability to fuzz most of NetBSD syscalls. In the final coding period of GSoC. I plan to analyse the crashes that were found until now. Integrate sanitizers, try and find more bugs and finally wrap up neatly with detailed documentation. > Last but not least, I would like to thank my mentor, Kamil Rytarowski for helping me through the process and guiding me. It has been a wonderful learning experience so far! Exploiting a no-name freebsd kernel vulnerability A new patch has been recently shipped in FreeBSD kernels to fix a vulnerability (cve-2019-5602) present in the cdrom device. In this post, we will introduce the bug and discuss its exploitation on pre/post-SMEP FreeBSD revisions. > A closer look at the commit 6bcf6e3 shows that when invoking the CDIOCREADSUBCHANNEL_SYSSPACE ioctl, data are copied with bcopy instead of the copyout primitive. This endows a local attacker belonging to the operator group with an arbitrary write primitive in the kernel memory. [Allan and Benedicts Conference Gear Breakdown] Benedict

OPNsense 19.7.1 is out, ZFS on Linux still has annoying issues with ARC size, Hammer2 is now default, NetBSD audio – an application perspective, new FreeNAS Mini, and more. Headlines OPNsense 19.7.1 We do not wish to keep you from enjoying your summer time, but this is a recommended security update enriched with reliability fixes for the new 19.7 series. Of special note are performance improvements as well as a fix for a longstanding NAT before IPsec limitation. Full patch notes: system: do not create automatic copies of existing gateways system: do not translate empty tunables descriptions system: remove unwanted form action tags system: do not include Syslog-ng in rc.freebsd handler system: fix manual system log stop/start/restart system: scoped IPv6 "%" could confuse mwexecf(), use plain mwexec() instead system: allow curl-based downloads to use both trusted and local authorities system: fix group privilege print and correctly redirect after edit system: use cached address list in referrer check system: fix Syslog-ng search stats firewall: HTML-escape dynamic entries to display aliases firewall: display correct IP version in automatic rules firewall: fix a warning while reading empty outbound rules configuration firewall: skip illegal log lines in live log interfaces: performance improvements for configurations with hundreds of interfaces reporting: performance improvements for Python 3 NetFlow aggregator rewrite dhcp: move advanced router advertisement options to correct config section ipsec: replace global array access with function to ensure side-effect free boot ipsec: change DPD action on start to "dpdaction = restart" ipsec: remove already default "dpdaction = none" if not set ipsec: use interface IP address in local ID when doing NAT before IPsec web proxy: fix database reset for Squid 4 by replacing use of ssl_crtd with security_file_certgen plugins: os-acme-client 1.24[1] plugins: os-bind 1.6[2] plugins: os-dnscrypt-proxy 1.5[3] plugins: os-frr now restricts characters BGP prefix-list and route-maps[4] plugins: os-google-cloud-sdk 1.0[5] ports: curl 7.65.3[6] ports: monit 5.26.0[7] ports: openssh 8.0p1[8] ports: php 7.2.20[9] ports: python 3.7.4[10] ports: sqlite 3.29.0[11] ports: squid 4.8[12] Stay safe and hydrated, Your OPNsense team ZFS on Linux still has annoying issues with ARC size One of the frustrating things about operating ZFS on Linux is that the ARC size is critical but ZFS's auto-tuning of it is opaque and apparently prone to malfunctions, where your ARC will mysteriously shrink drastically and then stick there. Linux's regular filesystem disk cache is very predictable; if you do disk IO, the cache will relentlessly grow to use all of your free memory. This sometimes disconcerts people when free reports that there's very little memory actually free, but at least you're getting value from your RAM. This is so reliable and regular that we generally don't think about 'is my system going to use all of my RAM as a disk cache', because the answer is always 'yes'. (The general filesystem cache is also called the page cache.) This is unfortunately not the case with the ZFS ARC in ZFS on Linux (and it wasn't necessarily the case even on Solaris). ZFS has both a current size and a 'target size' for the ARC (called 'c' in ZFS statistics). When your system boots this target size starts out as the maximum allowed size for the ARC, but various events afterward can cause it to be reduced (which obviously limits the size of your ARC, since that's its purpose). In practice, this reduction in the target size is both pretty sticky and rather mysterious (as ZFS on Linux doesn't currently expose enough statistics to tell why your ARC target size shrunk in any particular case). The net effect is that the ZFS ARC is not infrequently quite shy and hesitant about using memory, in stark contrast to Linux's normal filesystem cache. The default maximum ARC size starts out as only half of your RAM (unlike the regular filesystem cache, which will use all of it), and then it shrinks from there, sometimes very significantly, and once shrunk it only recovers slowly (if at all). News Roundup Hammer2 is now default commit a49112761c919d42d405ec10252eb0553662c824 Author: Matthew Dillon <dillon at apollo.backplane.com> Date: Mon Jun 10 17:53:46 2019 -0700 installer - Default to HAMMER2 * Change the installer default from HAMMER1 to HAMMER2. * Adjust the nrelease build to print the location of the image files when it finishes. Summary of changes: nrelease/Makefile | 2 +- usr.sbin/installer/dfuibe_installer/flow.c | 20 ++++++++++---------- 2 files changed, 11 insertions(+), 11 deletions(-) http://gitweb.dragonflybsd.org/dragonfly.git/commitdiff/a49112761c919d42d405ec10252eb0553662c824 NetBSD audio – an application perspective NetBSD audio – an application perspective ... or, "doing it natively, because we can&quo

DragonFlyBSD Project Update - colo upgrade, future trends, resuming ZFS send, realtime bandwidth terminal graph visualization, fixing telnet fixes, a chapter from the FBI’s history with OpenBSD and an OpenSSH vuln, and more. Headlines DragonFlyBSD Project Update - colo upgrade, future trends For the last week I've been testing out a replacement for Monster, our 48-core opteron server. The project will be removing Monster from the colo in a week or two and replacing it with three machines which together will use half the power that Monster did alone. The goal is to clear out a little power budget in the colo and to really beef-up our package-building capabilities to reduce the turn-around time needed to test ports syncs and updates to the binary package system. Currently we use two blades to do most of the building, plus monster sometimes. The blades take almost a week (120 hours+) to do a full synth run and monster takes around 27.5 hours. But we need to do three bulk builds more or less at the same time... one for the release branch, one for the development branch, and one for staging updates. It just takes too long and its been gnawing at me for a little while. Well, Zen 2 to the rescue! These new CPUs can take ECC, there's actually an IPMI mobo available, and they are fast as hell and cheap for what we get. The new machines will be two 3900X based servers, plus a dual-xeon system that I already had at home. The 3900X's can each do a full synth run in 24.5 hours and the Xeon can do it in around 31 hours. Monster will be retired. And the crazy thing about this? Monster burns 1000W going full bore. Each of the 3900X servers burns 160W and the Xeon burns 200W. In otherwords, we are replacing 1000W with only 520W and getting roughly 6x the performance efficiency in the upgrade. This tell you just how much more power-efficient machines have become in the last 9 years or so. > This upgrade will allow us to do full builds for both release and dev in roughly one day instead of seven days, and do it without interfering with staging work that might be happening at the same time. Future trends - DragonFlyBSD has reached a bit of a cross-roads. With most of the SMP work now essentially complete across the entire system the main project focus is now on supplying reliable binary ports for release and developer branches, DRM (GPU) support and other UI elements to keep DragonFlyBSD relevant on workstations, and continuing Filesystem work on HAMMER2 to get multi-device and clustering going. Resuming ZFS send One of the amazing functionalities of ZFS is the possibility of sending a whole dataset from one place to another. This mechanism is amazing to create backups of your ZFS based machines. Although, there were some issues with this functionality for a long time when a user sent a big chunk of data. What if you would do that over the network and your connection has disappeared? What if your machine was rebooted as you are sending a snapshot? For a very long time, you didn't have any options - you had to send a snapshot from the beginning. Now, this limitation was already bad enough. However, another downside of this approach was that all the data which you already send was thrown away. Therefore, ZFS had to go over all this data and remove them from the dataset. Imagine the terabytes of data which you sent via the network was thrown away because as you were sending the last few bytes, the network went off. In this short post, I don't want to go over the whole ZFS snapshot infrastructure (if you think that such a post would be useful, please leave a comment). Now, to get back to the point, this infrastructure is used to clone the datasets. Some time ago a new feature called “Resuming ZFS send” was introduced. That means that if there was some problem with transmitting the dataset from one point to another you could resume it or throw them away. But the point is, that yes, you finally have a choice. News Roundup Realtime bandwidth terminal graph visualization If for some reasons you want to visualize your bandwidth traffic on an interface (in or out) in a terminal with a nice graph, here is a small script to do so, involving ttyplot, a nice software making graphics in a terminal. The following will works on OpenBSD. You can install ttyplot by pkg_add ttyplot as root, ttyplot package appeared since OpenBSD 6.5. fixing telnet fixes There’s a FreeBSD commit to telnet. fix a couple of snprintf() buffer overflows. It’s received a bit of attention for various reasons, telnet in 2019?, etc. I thought I’d take a look. Here’s a few random observations. The first line is indented with spaces while the others use tabs. The correct type for string length is size_t not unsigned int. sizeof(char) is always one. There’s no need to multiply by it. If you do need to multiply by a size, this is an unsafe pattern. Use calloc or something similar. (OpenBSD provides reallocarray to avoid zeroing cost of calloc.) Return v

Replacing a (silently) failing disk in a ZFS pool, OPNsense 19.7 RC1 released, implementing DRM ioctl support for NetBSD, High quality/low latency VOIP server with umurmur/Mumble on OpenBSD, the PDP-7 where Unix began, LLDB watchpoints, and more. Headlines Replacing a (silently) failing disk in a ZFS pool Maybe I can’t read, but I have the feeling that official documentations explain every single corner case for a given tool, except the one you will actually need. My today’s struggle: replacing a disk within a FreeBSD ZFS pool. What? there’s a shitton of docs on this topic! Are you stupid? I don’t know, maybe. Yet none covered the process in a simple, straight and complete manner. OPNsense 19.7 RC1 released Hi there, For four and a half years now, OPNsense is driving innovation through modularising and hardening the open source firewall, with simple and reliable firmware upgrades, multi-language support, HardenedBSD security, fast adoption of upstream software updates as well as clear and stable 2-Clause BSD licensing. We thank all of you for helping test, shape and contribute to the project! We know it would not be the same without you. Download links, an installation guide[1] and the checksums for the images can be found below as well. News Roundup Implementation of DRM ioctl Support for NetBSD kernel What is DRM ioctl ? Ioctls are input/output control system calls and DRM stands for direct rendering manager The DRM layer provides several services to graphics drivers, many of them driven by the application interfaces it provides through libdrm, the library that wraps most of the DRM ioctls. These include vblank event handling, memory management, output management, framebuffer management, command submission & fencing, suspend/resume support, and DMA services. Native DRM ioctl calls NetBSD was able to make native DRM ioctl calls with hardware rendering once xorg and proper mesa packages where installed. We used the glxinfo and glxgears applications to test this out. High quality / low latency VOIP server with umurmur/Mumble on OpenBSD Discord users keep telling about their so called discord server, which is not dedicated to them at all. And Discord has a very bad quality and a lot of voice distorsion. Why not run your very own mumble server with high voice quality and low latency and privacy respect? This is very easy to setup on OpenBSD! Mumble is an open source voip client, it has a client named Mumble (available on various operating system) and at least Android, the server part is murmur but there is a lightweight server named umurmur. People authentication is done through certificate generated locally and automatically accepted on a server, and the certificate get associated with a nickname. Nobody can pick the same nickname as another person if it’s not the same certificate. TMWL June’19 — JS Fetch API, scheduling in Spring, thoughts on Unix Unix — going back to the roots From time to time, I like to review my knowledge in a certain area, even when I feel like I know a lot about it already. I go back to the basics and read tutorials, manuals, books or watch interesting videos. I’ve been using macOS for a couple of years now, previously being a linux user for some (relatively short) time. Both these operating systems have a common ancestor — Unix. While I’m definitely not an expert, I feel quite comfortable using linux & macOS — I understand the concepts behind the system architecture, know a lot of command line tools & navigate through the shell without a hassle. So-called unix philosophy is also close to my heart. I always feel like there’s more I could squeeze out of it. Recently, I found that book titled “Unix for dummies, 5th edition” which was published back in… 2004. Feels literally like AGES in the computer-related world. However, it was a great shot — the book starts with the basics, providing some brief history of Unix and how it came to life. It talks a lot about the structure of the system and where certain pieces fit (eg. “standard” set of tools), and how to understand permissions and work with files & directories. There’s even a whole chapter about shell-based text editors like Vi and Emacs! Despite the fact that I am familiar with most of these, I could still find some interesting pieces & tools that I either knew existed (but never had a chance to use), or even haven’t ever heard of. And almost all of these are still valid in the modern “incarnations” of Unix’s descendants: Linux and macOS. The book also talks about networking, surfing the web & working with email. It’s cute to see pictures of those old browsers rendering “ancient” Internet websites, but hey — this is how it looked like no more than fifteen years ago! I can really recommend this book to anyone working on modern macOS or Linux — you will certainly find some interesting pieces. Especially if you like to go back to the roots from time to time as I do! ThePDP-7 Where Unix Began In preparation for a ta

FreeBSD 11.3 has been released, OpenBSD workstation, write your own fuzzer for the NetBSD kernel, Exploiting FreeBSD-SA-19:02.fd, streaming to twitch using OpenBSD, 3 different ways of dumping hex contents of a file, and more. Headlines FreeBSD 11.3-RELEASE Announcement The FreeBSD Release Engineering Team is pleased to announce the availability of FreeBSD 11.3-RELEASE. This is the fourth release of the stable/11 branch. Some of the highlights: The clang, llvm, lld, lldb, and compiler-rt utilities as well as libc++ have been updated to upstream version 8.0.0. The ELF Tool Chain has been updated to version r3614. OpenSSL has been updated to version 1.0.2s. The ZFS filesystem has been updated to implement parallel mounting. The loader(8) has been updated to extend geli(8) support to all architectures. The pkg(8) utility has been updated to version 1.10.5. The KDE desktop environment has been updated to version 5.15.3. The GNOME desktop environment has been updated to version 3.28. The kernel will now log the jail(8) ID when logging a process exit. Several feature additions and updates to userland applications. Several network driver firmware updates. Warnings for features deprecated in future releases will now be printed on all FreeBSD versions. Warnings have been added for IPSec algorithms deprecated in RFC 8221. Deprecation warnings have been added for weaker algorithms when creating geli(8) providers. And more... OpenBSD Is Now My Workstation Why OpenBSD? Simply because it is the best tool for the job for me for my new-to-me Lenovo Thinkpad T420. Additionally, I do care about security and non-bloat in my personal operating systems (business needs can have different priorities, to be clear). I will try to detail what my reasons are for going with OpenBSD (instead of GNU/Linux, NetBSD, or FreeBSD of which I’m comfortable using without issue), challenges and frustrations I’ve encountered, and what my opinions are along the way. Disclaimer: in this post, I’m speaking about what is my opinion, and I’m not trying to convince you to use OpenBSD or anything else. I don’t truly care, but wanted to share in case it could be useful to you. I do hope you give OpenBSD a shot as your workstation, especially if it has been a while. A Bit About Me and OpenBSD I’m not new to OpenBSD, to be clear. I’ve been using it off and on for over 20 years. The biggest time in my life was the early 2000s (I was even the Python port maintainer for a bit), where I not only used it for my workstation, but also for production servers and network devices. I just haven’t used it as a workstation (outside of a virtual machine) in over 10 years, but have used it for servers. Workstation needs, especially for a primary workstation, are greatly different and the small things end up mattering most. News Roundup Write your own fuzzer for NetBSD kernel! [Part 1] How Fuzzing works? The dummy Fuzzer. The easy way to describe fuzzing is to compare it to the process of unit testing a program, but with different input. This input can be random, or it can be generated in some way that makes it unexpected form standard execution perspective. The simplest 'fuzzer' can be written in few lines of bash, by getting N bytes from /dev/rand, and putting them to the program as a parameter. Coverage and Fuzzing What can be done to make fuzzing more effective? If we think about fuzzing as a process, where we place data into the input of the program (which is a black box), and we can only interact via input, not much more can be done. However, programs usually process different inputs at different speeds, which can give us some insight into the program's behavior. During fuzzing, we are trying to crash the program, thus we need additional probes to observe the program's behaviour. Additional knowledge about program state can be exploited as a feedback loop for generating new input vectors. Knowledge about the program itself and the structure of input data can also be considered. As an example, if the input data is in the form of HTML, changing characters inside the body will probably cause less problems for the parser than experimenting with headers and HTML tags. For open source programs, we can read the source code to know what input takes which execution path. Nonetheless, this might be very time consuming, and it would be much more helpful if this can be automated. As it turns out, this process can be improved by tracing coverage of the execution vBSDcon - CFP - Call for Papers ends July 19th You can submit your proposal at https://easychair.org/conferences/?conf=vbsdcon2019 The talks will have a very strong technical content bias. Proposals of a business development or marketing nature are not appropriate for this venue. If you are doing something interesting with a BSD operating system, please submit a proposal. Whether you are developing a very complex system using BSD as the foundation, or helping others and have a story to tell about how BSD

Am5x86 based retro UNIX build log, setting up services in a FreeNAS Jail, first taste of DragonflyBSD, streaming Netflix on NetBSD, NetBSD on the last G4 Mac mini, Hammer vs Hammer2, and more. Headlines Polprog's Am5x86 based retro UNIX build log I have recently acquired an Am5x86 computer, in a surprisingly good condition. This is an ongoing project, check this page often for updates! I began by connecting a front panel. The panel came from a different chassis and is slightly too wide, so I had to attach it with a couple of zip-ties. However, that makes it stick out from the PC front at an angle, allowing easy access when the computer sits at the floor - and thats where it is most of the time. It's not that bad, to be honest, and its way easier to access than it would be, if mounted vertically There is a mains switch on the front panel because the computer uses an older style power supply. Those power supplies instead of relying on a PSON signal, like modern ATX supplies, run a 4 wire cable to a mains switch. The cable carries live and neutral both ways, and the switch keys in or out the power. The system powers on as soon as the switch is enabled. Originally there was no graphics card in it. Since a PC will not boot with out a GPU, I had to find one. The mainboard only has PCI and ISA slots, and all the GPUs I had were AGP. Fortunately, I bought a PCI GPU hoping it would solve my issue... However the GPU turned out to be faulty. It took me some time to repair it. I had to repair a broken trace leading to one of the EEPROM pins, and replace a contact in the EEPROM's socket. Then I replaced all the electrolytic capacitors on it, and that fixed it for good. Having used up only one of the three PCI slots, I populated the remaining pair with two ethernet cards. I still have a bunch of ISA slots available, but I have nothing to install there. Yet. See the article for the rest of the writeup Setting up services in a FreeNAS Jail This piece demonstrates the setup of a server service in a FreeNAS jail and how to share files with a jail using Apache 2.4 as an example. Jails are powerful, self-contained FreeBSD environments with separate network settings, package management, and access to thousands of FreeBSD application packages. Popular packages such as Apache, NGINX, LigHTTPD, MySQL, and PHP can be found and installed with the pkg search and pkg install commands. This example shows creating a jail, installing an Apache web server, and setting up a simple web page. NOTE: Do not directly attach FreeNAS to an external network (WAN). Use port forwarding, proper firewalls and DDoS protections when using FreeNAS for external web sites. This example demonstrates expanding the functionality of FreeNAS in an isolated LAN environment. News Roundup First taste of DragonflyBSD Last week, I needed to pick a BSD Operating System which supports NUMA to do some testing, so I decided to give Dragonfly BSD a shot. Dragonfly BSDonly can run on X86_64 architecture, which reminds me of Arch Linux, and after some tweaking, I feel Dragonfly BSD may be a “developer-friendly” Operating System, at least for me. I mainly use Dragonfly BSD as a server, so I don’t care whether GUI is fancy or not. But I have high requirements of developer tools, i.e., compiler and debugger. The default compiler of Dragonfly BSD is gcc 8.3, and I can also install clang 8.0.0 from package. This means I can test state-of-the-art features of compilers, and it is really important for me. gdb‘s version is 7.6.1, a little lag behind, but still OK. Furthermore, the upgradation of Dragonfly BSD is pretty simple and straightforward. I followed document to upgrade my Operating System to 5.6.0 this morning, just copied and pasted, no single error, booted successfully. Streaming Netflix on NetBSD Here's a step-by-step guide that allows streaming Netflix media on NetBSD using a intel-haxm accelerated QEMU vm. Heads-up! Sound doesn't work, but everything else is fine. Please read the rest of this thread for a solution to this!! “Sudo Mastery 2nd Edition” cover art reveal I’m about halfway through the new edition of Sudo Mastery. Assuming nothing terrible happens, should have a complete first draft in four to six weeks. Enough stuff has changed in sudo that I need to carefully double-check every single feature. (I’m also horrified by the painfully obsolete versions of sudo shipped in the latest versions of CentOS and Debian, but people running those operating systems are already accustomed to their creaky obsolescence.) But the reason for this blog post? I have Eddie Sharam’s glorious cover art. My Patronizers saw it last month, so now the rest of you get a turn. NetBSD on the last G4 Mac mini I'm a big fan of NetBSD. I've run it since 2000 on a Mac IIci (of course it's still running it) and I ran it for several years on a Power Mac 7300 with a G3 card which was the second incarnation of the Floodgap gopher server. Today I also still run it o

Website protection with OPNsense, FreeBSD Support Pull Request for ZFS-on-Linux, How much has Unix changed, Porting Wine to amd64 on NetBSD, FreeBSD Enterprise 1 PB Storage, the death watch for X11 has started, and more. Headlines Website protection with OPNsense with nginx plugin OPNsense become a strong full featured Web Application Firewall (WAF) The OPNsense security platform can help you to protect your network and your webservers with the nginx plugin addition. In old days, install an open source firewall was a very trick task, but today it can be done with few clicks (or key strokes). In this article I'll not describe the detailed OPNsense installation process, but you can watch this video that was extracted from my OPNsense course available in Udemy. The video is in portuguese language, but with the translation CC Youtube feature you may be able to follow it without problems (if you don't are a portuguese speaker ofcourse) :-) See the article for the rest of the writeup FreeBSD Support Pull Request against the ZFS-on-Linux repo This pull request integrates the sysutils/openzfs port’s sources into the upstream ZoL repo > Adding FreeBSD support to ZoL will make it easier to move changes back and forth between FreeBSD and Linux > Refactor tree to separate out Linux and FreeBSD specific code > import FreeBSD's SPL > add ifdefs in common code where it made more sense to do so than duplicate the code in separate files > Adapted ZFS Test Suite to run on FreeBSD and all tests that pass on ZoL passing on ZoF The plan to officially rename the common repo from ZFSonLinux to OpenZFS was announced at the ZFS Leadership Meeting on June 25th Video of Leadership Meeting Meeting Agenda and Notes This will allow improvements made on one OS to be made available more easily (and more quickly) to the other platforms For example, mav@’s recent work: Add wakeup_any(), cheaper version of wakeup_one() for taskqueue(9) > As result, on 72-core Xeon v4 machine sequential ZFS write to 12 ZVOLs with 16KB block size spend 34% less time in wakeup_any() and descendants then it was spending in wakeup_one(), and total write throughput increased by ~10% with the same as before CPU usage. News Roundup Episode 5 Notes - How much has UNIX changed? UNIX-like systems have dominated computing for decades, and with the rise of the internet and mobile devices their reach has become even larger. True, most systems now use more modern OSs like Linux, but how much has the UNIX-like landscape changed since the early days? So, my question was this: how close is a modern *NIX userland to some of the earliest UNIX releases? To do this I'm going to compare a few key points of a modern Linux system with the earliest UNIX documentation I can get my hands on. The doc I am going to be covering(https://www.tuhs.org/Archive/Distributions/Research/Dennis_v1/UNIX_ProgrammersManual_Nov71.pdf) is from November 1971, predating v1 of the system. I think the best place to start this comparison is to look at one of the highest-profile parts of the OS, that being the file system. Under the hood modern EXT file systems are completely different from the early UNIX file systems. However, they are still presented in basically the same way, as a heirerarchicat structure of directories with device files. So paths still look identical, and navigating the file system still functions the same. Often used commands like ls, cp, mv, du, and df function the same. So are mount and umount. But, there are some key differences. For instance, cd didn't exist, yet instead chdir filled its place. Also, chmod is somewhat different. Instead of the usual 3-digit octal codes for permissions, this older version only uses 2 digits. Really, that difference is due to the underlying file system using a different permission set than modern systems. For the most part, all the file handling is actually pretty close to a Linux system from 2019. See the article for the rest of the writeup Porting Wine to amd64 on NetBSD I have been working on porting Wine to amd64 on NetBSD as a GSoC 2019 project. Wine is a compatibility layer which allows running Microsoft Windows applications on POSIX-complaint operating systems. This report provides an overview of the progress of the project during the first coding period. Initially, when I started working on getting Wine-4.4 to build and run on NetBSD i386 the primary issue that I faced was Wine displaying black windows instead of UI, and this applied to any graphical program I tried running with Wine. I suspected it , as it is related to graphics, to be an issue with the graphics driver or Xorg. Subsequently, I tried building modular Xorg, and I tried running Wine on it only to realize that Xorg being modular didn't affect it in the least. After having tried a couple of configurations, I realized that trying to hazard out every other probability is going to take an awful lot of time that I didn't have. This motivat

DragonflyBSD 5.6 is out, OpenBSD Vulkan Support, bad utmp implementations in glibc and FreeBSD, OpenSSH protects itself against Side Channel attacks, ZFS vs OpenZFS, and more. Headlines DragonflyBSD 5.6 is out Version 5.6.0 released 17 June 2019 Version 5.6.1 released 19 June 2019 Big-ticket items Improved VM Informal test results showing the changes from 5.4 to 5.6 are available. Reduce stalls in the kernel vm_page_alloc() code (vm_page_list_find()). Improve page allocation algorithm to avoid re-iterating the same queues as the search is widened. Add a vm_page_hash*() API that allows the kernel to do heuristical lockless lookups of VM pages. Change vm_hold() and vm_unhold() semantics to not require any spin-locks. Change vm_page_wakeup() to not require any spin-locks. Change wiring vm_page's no longer manipulates the queue the page is on, saving a lot of overhead. Instead, the page will be removed from its queue only if the pageout demon encounters it. This allows pages to enter and leave the buffer cache quickly. Refactor the handling of fictitious pages. Remove m->md.pv_list entirely. VM pages in mappings no longer allocate pv_entry's, saving an enormous amount of memory when multiple processes utilize large shared memory maps (e.g. postgres database cache). Refactor vm_object shadowing, disconnecting the backing linkages from the vm_object itself and instead organizing the linkages in a new structure called vm_map_backing which hangs off the vm_map_entry. pmap operations now iterate vm_map_backing structures (rather than spin-locked page lists based on the vm_page and pv_entry's), and will test/match operations against the PTE found in the pmap at the requisite location. This doubles VM fault performance on shared pages and reduces the locking overhead for fault and pmap operations. Simplify the collapse code, removing most of the original code and replacing it with simpler per-vm_map_entry optimizations to limit the shadow depth. DRM Major updates to the radeon and ttm (amd support code) drivers. We have not quite gotten the AMD support up to the more modern cards or Ryzen APUs yet, however. Improve UEFI framebuffer support. A major deadlock has been fixed in the radeon/ttm code. Refactor the startup delay designed to avoid conflicts between the i915 driver initialization and X startup. Add DRM_IOCTL_GET_PCIINFO to improve mesa/libdrm support. Fix excessive wired memory build-ups. Fix Linux/DragonFly PAGE_MASK confusion in the DRM code. Fix idr_*() API bugs. HAMMER2 The filesystem sync code has been rewritten to significantly improve performance. Sequential write performance also improved. Add simple dependency tracking to prevent directory/file splits during create/rename/remove operations, for better consistency after a crash. Refactor the snapshot code to reduce flush latency and to ensure a consistent snapshot. Attempt to pipeline the flush code against the frontend, improving flush vs frontend write concurrency. Improve umount operation. Fix an allocator race that could lead to corruption. Numerous other bugs fixed. Improve verbosity of CHECK (CRC error) console messages. OpenBSD Vulkan Support Somewhat surprisingly, OpenBSD has added the Vulkan library and ICD loader support as their newest port. This new graphics/vulkan-loader port provides the generic Vulkan library and ICD support that is the common code for Vulkan implementations on the system. This doesn't enable any Vulkan hardware drivers or provide something new not available elsewhere, but is rare seeing Vulkan work among the BSDs. There is also in ports the related components like the SPIR-V headers and tools, glsllang, and the Vulkan tools and validation layers. This is of limited usefulness, at least for the time being considering OpenBSD like the other BSDs lag behind in their DRM kernel driver support that is ported over from the mainline Linux kernel tree but generally years behind the kernel upstream. Particularly with Vulkan, newer kernel releases are needed for some Vulkan features as well as achieving decent performance. The Vulkan drivers of relevance are the open-source Intel ANV Vulkan driver and Radeon RADV drivers, both of which are in Mesa though we haven't seen any testing results to know how well they would work if at all currently on OpenBSD, but they're at least in Mesa and obviously open-source. A note: The BSDs are no longer that far behind. FreeBSD 12.0 uses DRM from Linux 4.16 (April 2018), and the drm-devel port is based on Linux 5.0 (March 2019) OpenBSD -current as of April 2019 uses DRM from Linux 4.19.34 *** News Roundup Bad utmp implementations in glibc and freebsd I recently released another version – 0.5.0 – of Dinit, the service manager / init system. There were a number of minor improvements, including to the build system (just running “make” or “gmake” should be enough on any of the systems which have a pre-defined configuration, no need to edit mconfig by hand), but the main f

OpenZFS-kmod port available, using blacklistd with NPF as fail2ban replacement, ZFS raidz expansion alpha preview 1, audio VU-meter increases CO2 footprint rant, XSAVE and compat32 kernel work for LLDB, where icons for modern X applications come from, and more. Headlines ZFSonFreeBSD ports renamed OpenZFS The ZFS on FreeBSD project has renamed the userland and kernel ports from zol and zol-kmod to openzfs and openzfs-kmod The new versions from this week are IOCTL compatible with the command line tools in FreeBSD 12.0, so you can use the old userland with the new kernel module (although obviously not the new features) With the renaming it is easier to specify which kernel module you want to load in /boot/loader.conf: > zfs_load=”YES” or > openzfs_load=”YES” To load traditional or the newer version of ZFS The kmod still requires FreeBSD 12-stable or 13-current because it depends on the newer crypto support in the kernel for the ZFS native encryption feature. Allan is looking at ways to work around this, but it may not be practical. We would like to do an unofficial poll on how people would the userland to co-exist. Add a suffix to the new commands in /usr/local (zfs.new zpool.new or whatever). One idea i’ve had is to move the zfs and zpool commands to /libexec and make /sbin/zfs and /sbin/zpool a switcher script, that will call the base or ports version based on a config file (or just based on if the port is installed) For testing purposes, generally you should be fine as long as you don’t run ‘zpool upgrade’, which will make your pool only importable using the newer ZFS. For extra safety, you can create a ‘zpool checkpoint’, which will allow you to undo any changes that are made to the pool during your testing with the new openzfs tools. Note: the checkpoint will undo EVERYTHING. So don’t save new data you want to keep. Note: Checkpoints disable all freeing operations, to prevent any data from being overwritten so that you can re-import at the checkpoint and undo any operation (including zfs destroy-ing a dataset), so also be careful you don’t run out of space during testing. Please test and provide feedback. How to use blacklistd(8) with NPF as a fail2ban replacement About blacklistd(8) blacklistd(8) provides an API that can be used by network daemons to communicate with a packet filter via a daemon to enforce opening and closing ports dynamically based on policy. The interface to the packet filter is in /libexec/blacklistd-helper (this is currently designed for npf) and the configuration file (inspired from inetd.conf) is in etc/blacklistd.conf Now, blacklistd(8) will require bpfjit(4) (Just-In-Time compiler for Berkeley Packet Filter) in order to properly work, in addition to, naturally, npf(7) as frontend and syslogd(8), as a backend to print diagnostic messages. Also remember npf shall rely on the npflog* virtual network interface to provide logging for tcpdump() to use. Unfortunately (dont' ask me why :P) in 8.1 all the required kernel components are still not compiled by default in the GENERIC kernel (though they are in HEAD), and are rather provided as modules. Enabling NPF and blacklistd services would normally result in them being automatically loaded as root, but predictably on securelevel=1 this is not going to happen News Roundup [WIP] raidz expansion, alpha preview 1 Motivation and Context > This is a alpha-quality preview of RAID-Z expansion. This feature allows disks to be added one at a time to a RAID-Z group, expanding its capacity incrementally. This feature is especially useful for small pools (typically with only one RAID-Z group), where there isn't sufficient hardware to add capacity by adding a whole new RAID-Z group (typically doubling the number of disks). > For additional context as well as a design overview, see my short talk from the 2017 OpenZFS Developer Summit: slides video Rant: running audio VU-meter increases my CO2 footprint A couple months ago I noticed that the monitor on my workstation never power off anymore. Screensaver would go on, but DPMs (to do the poweroff) never kicked in. I grovels the output of various tools that display DPMS settings, which as usual in Xorg were useless. Everybody said DPMS is on with a timeout. I even wrote my own C program to use every available Xlib API call and even the xscreensaver library calls. (should make it available) No go, everybody says that DPMs is on, enabled and set on a timeout. Didn’t matter whether I let xscreeensaver do the job or just the X11 server. After a while I noticed that DPMS actually worked between starting my X11 server and starting all my clients. I have a minimal .xinitrc and start the actual session from a script, that is how I could notice. If I used a regular desktop login I wouldn’t have noticed. A server state bug was much more likely than a client bug. See the article for the rest... XSAVE and compat32 kernel work for LLDB Upstream describes LLDB as a next generation, high-performance

DragonFlyBSD's kernel optimizations pay off, differences between OpenBSD and Linux, NetBSD 2019 Google Summer of Code project list, Reducing that contention, fnaify 1.3 released, vmctl(8): CLI syntax changes, and things that Linux distributions should not do when packaging. Headlines DragonFlyBSD's Kernel Optimizations Are Paying Off DragonFlyBSD lead developer Matthew Dillon has been working on a big VM rework in the name of performance and other kernel improvements recently. Here is a look at how those DragonFlyBSD 5.5-DEVELOPMENT improvements are paying off compared to DragonFlyBSD 5.4 as well as FreeBSD 12 and five Linux distribution releases. With Dillon using an AMD Ryzen Threadripper system, we used that too for this round of BSD vs. Linux performance benchmarks. The work by Dillon on the VM overhaul and other changes (including more HAMMER2 file-system work) will ultimately culminate with the DragonFlyBSD 5.6 release (well, unless he opts for DragonFlyBSD 6.0 or so). These are benchmarks of the latest DragonFlyBSD 5.5-DEVELOPMENT daily ISO as of this week benchmarked across DragonFlyBSD 5.4.3 stable, FreeBSD 12.0, Ubuntu 19.04, Red Hat Enterprise Linux 8.0, Debian 9.9, Debian Buster, and CentOS 7 1810 as a wide variety of reference points both from newer and older Linux distributions. (As for no Clear Linux reference point for a speedy reference point, it currently has a regression with AMD + Samsung NVMe SSD support on some hardware, including this box, prohibiting the drive from coming up due to a presumed power management issue that is still being resolved.) With Matthew Dillon doing much of his development on an AMD Ryzen Threadripper system after he last year proclaimed the greatness of these AMD HEDT CPUs, for this round of testing I also used a Ryzen Threadripper 2990WX with 32 cores / 64 threads. Tests of other AMD/Intel hardware with DragonFlyBSD will come as the next stable release is near and all of the kernel work has settled down. For now it's mostly entertaining our own curiosity how well these DragonFlyBSD optimizations are paying off and how it's increasing the competition against FreeBSD 12 and Linux distributions. What are the differences between OpenBSD and Linux? Maybe you have been reading recently about the release of OpenBSD 6.5 and wonder, "What are the differences between Linux and OpenBSD?" I've also been there at some point in the past and these are my conclusions. They also apply, to some extent, to other BSDs. However, an important disclaimer applies to this article. This list is aimed at people who are used to Linux and are curious about OpenBSD. It is written to highlight the most important changes from their perspective, not the absolute most important changes from a technical standpoint. Please bear with me. A terminal is a terminal is a terminal Practical differences Security and system administration Why philosophical differences matter So what do I choose? How to try OpenBSD *** News Roundup NetBSD 2019 Google Summer of Code We are very happy to announce The NetBSD Foundation Google Summer of Code 2019 projects: Akul Abhilash Pillai - Adapting TriforceAFL for NetBSD kernel fuzzing Manikishan Ghantasala - Add KNF (NetBSD style) clang-format configuration Siddharth Muralee - Enhancing Syzkaller support for NetBSD Surya P - Implementation of COMPAT_LINUX and COMPAT_NETBSD32 DRM ioctls support for NetBSD kernel Jason High - Incorporation of Argon2 Password Hashing Algorithm into NetBSD Saurav Prakash - Porting NetBSD to HummingBoard Pulse Naveen Narayanan - Porting WINE to amd64 architecture on NetBSD The communiting bonding period - where students get in touch with mentors and community - started yesterday. The coding period will start from May 27 until August 19. Please welcome all our students and a big good luck to students and mentors! A big thank to Google and The NetBSD Foundation organization mentors and administrators! Looking forward to a great Google Summer of Code! Reducing that contention The opening keynote at EuroBSDCon 2016 predicted the future 10 years of BSDs. Amongst all the funny previsions, gnn@FreeBSD said that by 2026 OpenBSD will have its first implementation of SMP. Almost 3 years after this talk, that sounds like a plausible forecast... Why? Where are we? What can we do? Let's dive into the issue! State of affairs Most of OpenBSD's kernel still runs under a single lock, ze KERNEL_LOCK(). That includes most of the syscalls, most of the interrupt handlers and most of the fault handlers. Most of them, not all of them. Meaning we have collected & fixed bugs while setting up infrastructures and examples. Now this lock remains the principal responsible for the spin % you can observe in top(1) and systat(1). I believe that we opted for a difficult hike when we decided to start removing this lock from the bottom. As a result many SCSI & Network interrupt handlers as well as all Audio & USB ones can be

GPU passthrough on bhyve, confusion with used/free disk space on ZFS, OmniOS Community Edition, pfSense 2.4.4 Release p3, NetBSD 8.1 RC1, FreeNAS as your Server OS, and more. Headlines GPU Passthrough Reported Working on Bhyve Normally we cover news focused on KVM and sometimes Xen, but something very special has happened with their younger cousin in the BSD world, Bhyve. For those that don’t know, Bhyve (pronounced bee-hive) is the native hypervisor in FreeBSD. It has many powerful features, but one that’s been a pain point for some years now is VGA passthrough. Consumer GPUs have not been useable until very recently despite limited success with enterprise cards. However, Twitter user Michael Yuji found a workaround that enables passing through a consumer card to any *nix system configured to use X11: https://twitter.com/michael_yuji/status/1127136891365658625 All you have to do is add a line pointing the X server to the Bus ID of the passed card and the VM will boot, with acceleration and everything. He theorizes that this may not be possible on windows because of the way it looks for display devices, but it’s a solid start. As soon as development surrounding VGA passthrough matures on Bhyve, it will become a very attractive alternative to more common tools like Hyper-V and Qemu, because it makes many powerful features available in the host system like jails, boot environments, BSD networking, and tight ZFS integration. For example, you could potentially run your Router, NAS, preferred workstation OS and any number of other things in one box, and only have to spin up a single VM because of the flexibility afforded by jails over Linux-based containers. The user who found this workaround also announced they’d be writing it up at some point, so stay tuned for details on the process. It’s been slow going on Bhyve passthrough development for a while, but this new revelation is encouraging. We’ll be closely monitoring the situation and report on any other happenings. Confusion with used/free disk space in ZFS I use ZFS extensively. ZFS is my favorite file system. I write articles and give lectures about it. I work with it every day. In traditional file systems we use df(1) to determine free space on partitions. We can also use du(1) to count the size of the files in the directory. But it’s different on ZFS and this is the most confusing thing EVER. I always forget which tool reports what disk space usage! Every time somebody asks me, I need to google it. For this reason I decided to document it here - for myself - because if I can’t remember it at least I will not need to google it, as it will be on my blog, but maybe you will also benefit from this blog post if you have the same problem or you are starting your journey with ZFS. The understanding of how ZFS is uses space and how to determine which value means what is a crucial thing. I hope thanks to this article I will finally remember it! News Roundup OmniOS Community Edition The OmniOS Community Edition Association is proud to announce the general availability of OmniOS - r151030. OmniOS is published according to a 6-month release cycle, r151030 LTS takes over from r151028, published in November 2018; and since it is a LTS release it also takes over from r151022. The r151030 LTS release will be supported for 3 Years. It is the first LTS release published by the OmniOS CE Association since taking over the reins from OmniTI in 2017. The next LTS release is scheduled for May 2021. The old stable r151026 release is now end-of-life. See the release schedule for further details. This is only a small selection of the new features, and bug fixes in the new release; review the release notes for full details. If you upgrade from r22 and want to see all new features added since then, make sure to also read the release notes for r24, r26 and r28. The OmniOS team and the illumos community have been very active in creating new features and improving existing ones over the last 6 months. pfSense 2.4.4 Release p3 is available We are pleased to announce the release of pfSense® software version 2.4.4-p3, now available for new installations and upgrades! pfSense software version 2.4.4-p3 is a maintenance release, bringing a number of security enhancements as well as a handful of fixes for issues present in the 2.4.4-p2 release. pfSense 2.4.4-RELEASE-p3 updates and installation images are available now! To see a complete list of changes and find more detail, see the Release Notes. We had hoped to bring you this release a few days earlier, but given the announcement last Tuesday of the Intel Microarchitectural Data Sampling (MDS) issue, we did not have sufficient time to fully incorporate those corrections and properly test for release on Thursday. We felt that it was worth delaying for a few days, rather than making multiple releases within a week. Upgrade Notes Due to the significant nature of the changes in 2.4.4 and later, warnings and error messages, particularly from PHP

FreeBSD 11.3-beta 1 is out, BSDCan 2019 recap, OpenIndiana 2019.04 is out, Overview of ZFS Pools in FreeNAS, why open source firmware is important for security, a new Opnsense release, wireguard on OpenBSD, and more. Headlines FreeBSD 11.3-b1 is out BSDCan 2019 Recap We’re back from BSDCan and it was a packed week as always. It started with bhyvecon on Tuesday. Meanwhile, Benedict spent the whole day in productive meetings: annual FreeBSD Foundation board meeting and FreeBSD Journal editorial board meeting. On Wednesday, tutorials for BSDCan started as well as the FreeBSD Developer Summit. In the mornings, there were presentations in the big auditorium, while working groups about networking, failsafe bootcode, development web services, swap space management, and testing/CI were held. Friday had a similar format with an update from the FreeBSD core team and the “have, need, want” session for FreeBSD 13. In the afternoon, there were working groups about translation tools, package base, GSoC/Outreachy, or general hacking. Benedict held his Icinga tutorial in the afternoon with about 15 people attending. Devsummit presentation slides can be found on the wiki page and video recordings done by ScaleEngine are available on FreeBSD’s youtube channel. The conference program was a good mixture of sysadmin and tech talks across the major BSDs. Benedict saw the following talks: How ZFS snapshots really work by Matt Ahrens, 20 years in Jail by Michael W. Lucas, OpenZFS BOF session, the future of OpenZFS and FreeBSD, MQTT for system administrators by Jan-Piet Mens, and spent the rest of the time in between in the hallway track. Photos from the event are available on Ollivier Robert’s talegraph and Diane Bruce’s website for day 1, day 2, conference day 1, and conference day 2. Thanks to all the sponsors, supporters, organizers, speakers, and attendees for making this yet another great BSDCan. Next year’s BSDCan will be from June 2 - 6, 2020. OpenIndiana 2019.04 is out We have released a new OpenIndiana Hipster snapshot 2019.04. The noticeable changes: Firefox was updated to 60.6.3 ESR Virtualbox packages were added (including guest additions) Mate was updated to 1.22 IPS has received updates from OmniOS CE and Oracle IPS repos, including automatic boot environment naming Some OI-specific applications have been ported from Python 2.7/GTK 2 to Python 3.5/GTK 3 Quick Demo Video: https://www.youtube.com/watch?v=tQ0-fo3XNrg News Roundup Overview of ZFS Pools in FreeNAS FreeNAS uses the OpenZFS (ZFS) file system, which handles both disk and volume management. ZFS offers RAID options mirror, stripe, and its own parity distribution called RAIDZ that functions like RAID5 on hardware RAID. The file system is extremely flexible and secure, with various drive combinations, checksums, snapshots, and replication all possible. For a deeper dive on ZFS technology, read the ZFS Primer section of the FreeNAS documentation. SUGGEST LAYOUT attempts to balance usable capacity and redundancy by automatically choosing an ideal vdev layout for the number of available disks. The following vdev layout options are available when creating a pool: Stripe data is shared on two drives, similar to RAID0) Mirror copies data on two drives, similar to RAID1 but not limited to 2 disks) RAIDZ1 single parity similar to RAID5 RAIDZ2 double parity similar to RAID6 RAIDZ3 which uses triple parity and has no RAID equivalent Why OpenSource Firmware is Important for Security Roots of Trust The goal of the root of trust should be to verify that the software installed in every component of the hardware is the software that was intended. This way you can know without a doubt and verify if hardware has been hacked. Since we have very little to no visibility into the code running in a lot of places in our hardware it is hard to do this. How do we really know that the firmware in a component is not vulnerable or that is doesn’t have any backdoors? Well we can’t. Not unless it was all open source. Every cloud and vendor seems to have their own way of doing a root of trust. Microsoft has Cerberus, Google has Titan, and Amazon has Nitro. These seem to assume an explicit amount of trust in the proprietary code (the code we cannot see). This leaves me with not a great feeling. Wouldn’t it be better to be able to use all open source code? Then we could verify without a doubt that the code you can read and build yourself is the same code running on hardware for all the various places we have firmware. We could then verify that a machine was in a correct state without a doubt of it being vulnerable or with a backdoor. It makes me wonder what the smaller cloud providers like DigitalOcean or Packet have for a root of trust. Often times we only hear of these projects from the big three or five. OPNsense This update addresses several privilege escalation issues in the access control implementation and new memory disclosure issues in Intel CPUs. We would like to thank Arnaud Cordier

Running AIX on QEMU on Linux on Windows, your NAS fleet with TrueCommand, Unleashed 1.3 is available, LLDB: CPU register inspection support extension, V7 Unix programs often not written as expected, and more. Headlines Running AiX on QEMU on Linux on Windows YES it’s real! I’m using the Linux subsystem on Windows, as it’s easier to build this Qemu tree from source. I’m using Debian, but these steps will work on other systems that use Debian as a base. first thing first, you need to get your system with the needed pre-requisites to compile Great with those in place, now clone Artyom Tarasenko’s source repository Since the frame buffer apparently isn’t quite working just yet, I configure for something more like a text mode build. Now for me, GCC 7 didn’t build the source cleanly. I had to make a change to the file config-host.mak and remove all references to -Werror. Also I removed the sound hooks, as we won’t need them. Now you can build Qemu. Okay, all being well you now have a Qemu. Now following the steps from Artyom Tarasenko’s blog post, we can get started on the install! See article for rest of walkthrough. Take Command of Your NAS Fleet with TrueCommand Hundreds of thousands of FreeNAS and TrueNAS systems are deployed around the world, with many sites having dozens of systems. Managing multiple systems individually can be time-consuming. iXsystems has responded to the challenge by creating a “single pane of glass” application to simplify the scaling of data, drive management, and administration of iXsystems NAS platforms. We are proud to introduce TrueCommand. TrueCommand is a ZFS-aware management application that manages TrueNAS and FreeNAS systems. The public Beta of TrueCommand is available for download now. TrueCommand can be used with small iXsystems NAS fleets for free. Licenses can be purchased for large-scale deployments and enterprise support. TrueCommand expands on the ease of use and power of TrueNAS and FreeNAS systems with multi-system management and reporting. News Roundup Unleashed 1.3 Released This is the fourth release of Unleashed - an operating system fork of illumos. For more information about Unleashed itself and the download links, see our website. As one might expect, this release removes a few things. The most notable being the removal of ksh93 along with all its libs. As far as libc interfaces are concerned, a number of non-standard functions were removed. In general, they have been replaced by the standards-compliant versions. (getgrentr, fgetgrentr, getgrgidr, getgrnamr, ttynamer, getloginr, shmdt, sigwait, gethostname, putmsg, putpmsg, and getaddrinfo) Additionally, wordexp and wordfree have been removed from libc. Even though they are technically required by POSIX, software doesn't seem to use them. Because of the fragile implementation (shelling out), we took the OpenBSD approach and just removed them. The default compilation environment now includes XOPENSOURCE=700 and EXTENSIONS. Additionally, all applications now use 64-bit file offsets, making use of LARGEFILESOURCE, LARGEFILE64SOURCE, and FILEOFFSET_BITS unnecessary. Last but not least, nightly.sh is no more. In short, to build one simply runs 'make'. (See README for detailed build instructions.) Why Unleashed Why did we decide to fork illumos? After all, there are already many illumos distributions available to choose from. We felt we could do better than any of them by taking a more aggressive stance toward compatibility and reducing cruft from code and community interactions alike. LLDB: extending CPU register inspection support Upstream describes LLDB as a next generation, high-performance debugger. It is built on top of LLVM/Clang toolchain, and features great integration with it. At the moment, it primarily supports debugging C, C++ and ObjC code, and there is interest in extending it to more languages. In February, I have started working on LLDB, as contracted by the NetBSD Foundation. So far I've been working on reenabling continuous integration, squashing bugs, improving NetBSD core file support and updating NetBSD distribution to LLVM 8 (which is still stalled by unresolved regressions in inline assembly syntax). You can read more about that in my Mar 2019 report. In April, my main focus was on fixing and enhancing the support for reading and writing CPU registers. In this report, I'd like to shortly summarize what I have done, what I have learned in the process and what I still need to do. Future plans My work continues with the two milestones from last month, plus a third that's closely related: Add support for FPU registers support for NetBSD/i386 and NetBSD/amd64. Support XSAVE, XSAVEOPT, ... registers in core(5) files on NetBSD/amd64. Add support for Debug Registers support for NetBSD/i386 and NetBSD/amd64. The most important point right now is deciding on the format for passing the remaining registers, and implementing the missing ptrace interface kernel-side. The support for core files should follow

36 year old UFS bug fixed, a BSD for the road, automatic upgrades with OpenBSD, DTrace ext2fs support in FreeBSD, Dedicated SSH tunnel user, upgrading VMM VMs to OpenBSD 6.5, and more. Headlines 36+ year old bug in FFS/UFS discovered and patched This update eliminates a kernel stack disclosure bug in UFS/FFS directory entries that is caused by uninitialized directory entry padding written to the disk. When the directory entry is written to disk, it is written as a full 32bit entry, and the unused bytes were not initialized, so could possibly contain sensitive data from the kernel stack It can be viewed by any user with read access to that directory. Up to 3 bytes of kernel stack are disclosed per file entry, depending on the the amount of padding the kernel needs to pad out the entry to a 32 bit boundary. The offset in the kernel stack that is disclosed is a function of the filename size. Furthermore, if the user can create files in a directory, this 3 byte window can be expanded 3 bytes at a time to a 254 byte window with 75% of the data in that window exposed. The additional exposure is done by removing the entry, creating a new entry with a 4-byte longer name, extracting 3 more bytes by reading the directory, and repeating until a 252 byte name is created. This exploit works in part because the area of the kernel stack that is being disclosed is in an area that typically doesn't change that often (perhaps a few times a second on a lightly loaded system), and these file creates and unlinks themselves don't overwrite the area of kernel stack being disclosed. It appears that this bug originated with the creation of the Fast File System in 4.1b-BSD (Circa 1982, more than 36 years ago!), and is likely present in every Unix or Unix-like system that uses UFS/FFS. Amazingly, nobody noticed until now. This update also adds the -z flag to fsck_ffs to have it scrub the leaked information in the name padding of existing directories. It only needs to be run once on each UFS/FFS filesystem after a patched kernel is installed and running. Submitted by: David G. Lawrence dg@dglawrence.com So a patched kernel will no longer leak this data, and running the fsck_ffs -z command will erase any leaked data that may exist on your system OpenBSD commit with additional detail on mitigations The impact on OpenBSD is very limited: 1 - such stack bytes can be found in raw-device reads, from group operator. If you can read the raw disks you can undertake other more powerful actions. 2 - read(2) upon directory fd was disabled July 1997 because I didn't like how grep * would display garbage and mess up the tty, and applying vis(3) for just directory reads seemed silly. read(2) was changed to return 0 (EOF). Sep 2016 this was further changed to EISDIR, so you still cannot see the bad bytes. 3 - In 2013 when guenther adapted the getdents(2) directory-reading system call to 64-bit ino_t, the userland data format changed to 8-byte-alignment, making it incompatible with the 4-byte-alignment UFS on-disk format. As a result of code refactoring the bad bytes were not copied to userland. Bad bytes will remain in old directories on old filesystems, but nothing makes those bytes user visible. There will be no errata or syspatch issued. I urge other systems which do expose the information to userland to issue errata quickly, since this is a 254 byte infoleak of the stack which is great for ROP-chain building to attack some other bug. Especially if the kernel has no layout/link-order randomization ... NomadBSD, a BSD for the Road As regular It’s FOSS readers should know, I like diving into the world of BSDs. Recently, I came across an interesting BSD that is designed to live on a thumb drive. Let’s take a look at NomadBSD. NomadBSD is different than most available BSDs. NomadBSD is a live system based on FreeBSD. It comes with automatic hardware detection and an initial config tool. NomadBSD is designed to “be used as a desktop system that works out of the box, but can also be used for data recovery, for educational purposes, or to test FreeBSD’s hardware compatibility.” This German BSD comes with an OpenBox-based desktop with the Plank application dock. NomadBSD makes use of the DSB project. DSB stands for “Desktop Suite (for) (Free)BSD” and consists of a collection of programs designed to create a simple and working environment without needing a ton of dependencies to use one tool. DSB is created by Marcel Kaiser one of the lead devs of NomadBSD. Just like the original BSD projects, you can contact the NomadBSD developers via a mailing list. Version 1.2 Released NomadBSD recently released version 1.2 on April 21, 2019. This means that NomadBSD is now based on FreeBSD 12.0-p3. TRIM is now enabled by default. One of the biggest changes is that the initial command-line setup was replaced with a Qt graphical interface. They also added a Qt5 tool to install NomadBSD to your

FreeBSD ZFS vs. ZoL performance, Dragonfly 5.4.2 has been release, containing web services with iocell, Solaris 11.4 SRU8, Problem with SSH Agent forwarding, OpenBSD 6.4 to 6.5 upgrade guide, and more. Headlines FreeBSD ZFS vs. ZoL Performance, Ubuntu ZFS On Linux Reference With iX Systems having released new images of FreeBSD reworked with their ZFS On Linux code that is in development to ultimately replace their existing FreeBSD ZFS support derived from the code originally found in the Illumos source tree, here are some fresh benchmarks looking at the FreeBSD 12 performance of ZFS vs. ZoL vs. UFS and compared to Ubuntu Linux on the same system with EXT4 and ZFS. Using an Intel Xeon E3-1275 v6 with ASUS P10S-M WS motherboard, 2 x 8GB DDR4-2400 ECC UDIMMs, and Samsung 970 EVO Plus 500GB NVMe solid-state drive was used for all of this round of testing. Just a single modern NVMe SSD was used for this round of ZFS testing while as the FreeBSD ZoL code matures I'll test on multiple systems using a more diverse range of storage devices. FreeBSD 12 ZoL was tested using the iX Systems image and then fresh installs done of FreeBSD 12.0-RELEASE when defaulting to the existing ZFS root file-system support and again when using the aging UFS file-system. Ubuntu 18.04.2 LTS with the Linux 4.18 kernel was used when testing its default EXT4 file-system and then again when using the Ubuntu-ZFS ZoL support. Via the Phoronix Test Suite various BSD/Linux I/O benchmarks were carried out. Overall, the FreeBSD ZFS On Linux port is looking good so far and we are looking forward to it hopefully maturing in time for FreeBSD 13.0. Nice job to iX Systems and all of those involved, especially the ZFS On Linux project. Those wanting to help in testing can try the FreeBSD ZoL spins. Stay tuned for more benchmarks and on more diverse hardware as time allows and the FreeBSD ZoL support further matures, but so far at least the performance numbers are in good shape. DragonFlyBSD 5.4.2 is out Upgrading guide Here's the tag commit, for what has changed from 5.4.1 to 5.4.2 The normal ISO and IMG files are available for download and install, plus an uncompressed ISO image for those installing remotely. I uploaded them to mirror-master.dragonflybsd.org last night so they should be at your local mirror or will be soon. This version includes Matt's fix for the HAMMER2 corruption bug he identified recently. If you have an existing 5.4 system and are running a generic kernel, the normal upgrade process will work. > cd /usr/src > git pull > make buildworld. > make buildkernel. > make installkernel. > make installworld > make upgrade After your next reboot, you can optionally update your rescue system: > cd /usr/src > make initrd As always, make sure your packages are up to date: > pkg update > pkg upgrade News Roundup Containing web services with iocell I'm a huge fan of the FreeBSD jails feature. It is a great system for splitting services into logical units with all the performance of the bare metal system. In fact, this very site runs in its own jail! If this is starting to sound like LXC or Docker, it might surprise you to learn that OS-level virtualization has existed for quite some time. Kudos to the Linux folks for finally getting around to it. 😛 If you're interested in the history behind Jails, there is an excellent talk from Papers We Love on the subject: https://www.youtube.com/watch?v=hgN8pCMLI2U Getting started There are plenty of options when it comes to setting up the jail system. Ezjail and Iocage seem popular, or you could do things manually. Iocage was recently rewritten in python, but was originally a set of shell scripts. That version has since been forked under the name Iocell, and I think it's pretty neat, so this tutorial will be using Iocell. To start, you'll need the following: A FreeBSD install (we'll be using 11.0) The iocell package (available as a package, also in the ports tree) A ZFS pool for hosting the jails Once you have installed iocell and configured your ZFS pool, you'll need to run a few commands before creating your first jail. First, tell iocell which ZFS pool to use by issuing iocell activate $POOLNAME. Iocell will create a few datasets. As you can imagine, your jails are contained within the /iocell/jails dataset. The /iocell/releases dataset is used for storing the next command we need to run, iocell fetch. Iocell will ask you which release you'd like to pull down. Since we're running 11.0 on the host, pick 11.0-RELEASE. Iocell will download the necessary txz files and unpack them in /iocell/releases. See Article for the rest of the walkthrough. Oracle Solaris 11.4 SRU8 Today we are releasing the SRU 8 for Oracle Solaris 11.4. It is available via 'pkg update' from the support repository or by downloading the SRU from My Oracle Support Doc ID 2433412.1. This SRU introduces the following enhancements: Integration of 28060039 introduced an issue where any firmware update/query c

OpenBSD 6.5 has been released, mount ZFS datasets anywhere, help test upcoming NetBSD 9 branch, LibreSSL 2.9.1 is available, Bail Bond Denied Edition of FreeBSD Mastery: Jails, and one reason ed(1) was a good editor back in the days in this week’s episode. Headlines OpenBSD 6.5 Released Changelog Mirrors 6.5 Includes OpenSMTPD 6.5.0 LibreSSL 2.9.1 OpenSSH 8.0 Mandoc 1.14.5 Xenocara LLVM/Clang 7.0.1 (+ patches) GCC 4.2.1 (+ patches) and 3.3.6 (+ patches) Many pre-built packages for each architecture: aarch64: 9654 amd64: 10602 i386: 10535 Mount your ZFS datasets anywhere you want ZFS is very flexible about mountpoints, and there are many features available to provide great flexibility. When you create zpool maintank, the default mountpoint is /maintank. You might be happy with that, but you don’t have to be content. You can do magical things. Some highlights are: mount point can be inherited not all filesystems in a zpool need to be mounted each filesystem (directory) can have different ZFS characteristics In my case, let’s look at this new zpool I created earlier today and I will show you some very simple alternatives. This zpool use NVMe devices which should be faster than SSDs especially when used with multiple concurrent writes. This is my plan: run all the Bacula regression tests concurrently. News Roundup Branch for netbsd 9 upcoming, please help and test -current Folks, once again we are quite late for branching the next NetBSD release (NetBSD 9). Initially planned to happen early in February 2019, we are now approaching May and it is unlikely that the branch will happen before that. On the positive side, lots of good things landed in -current in between, like new Mesa, new jemalloc, lots of ZFS improvements - and some of those would be hard to pull up to the branch later. On the bad side we saw lots of churn in -current recently, and there is quite some fallout where we not even have a good overview right now. And this is where you can help: please test -current, on all the various machines you have especially interesting would be test results from uncommon architectures or strange combinations (like the sparc userland on sparc64 kernel issue I ran in yesterday) Please test, report success, and file PRs for failures! We will likely announce the real branch date on quite short notice, the likely next candidates would be mid may or end of may. We may need to do extra steps after the branch (like switch some architectures back to old jemalloc on the branch). However, the less difference between -current and the branch, the easier will the release cycle go. Our goal is to have an unprecedented short release cycle this time. But.. we always say that upfront. LibreSSL 2.9.1 Released We have released LibreSSL 2.9.1, which will be arriving in the LibreSSL directory of your local OpenBSD mirror soon. This is the first stable release from the 2.9 series, which is also included with OpenBSD 6.5 It includes the following changes and improvements from LibreSSL 2.8.x: API and Documentation Enhancements CRYPTO_LOCK is now automatically initialized, with the legacy callbacks stubbed for compatibility. Added the SM3 hash function from the Chinese standard GB/T 32905-2016. Added the SM4 block cipher from the Chinese standard GB/T 32907-2016. Added more OPENSSLNO* macros for compatibility with OpenSSL. Partial port of the OpenSSL ECKEYMETHOD API for use by OpenSSH. Implemented further missing OpenSSL 1.1 API. Added support for XChaCha20 and XChaCha20-Poly1305. Added support for AES key wrap constructions via the EVP interface. Compatibility Changes Added pbkdf2 key derivation support to openssl(1) enc. Changed the default digest type of openssl(1) enc to sha256. Changed the default digest type of openssl(1) dgst to sha256. Changed the default digest type of openssl(1) x509 -fingerprint to sha256. Changed the default digest type of openssl(1) crl -fingerprint to sha256. Testing and Proactive Security Added extensive interoperability tests between LibreSSL and OpenSSL 1.0 and 1.1. Added additional Wycheproof tests and related bug fixes. Internal Improvements Simplified sigalgs option processing and handshake signing algorithm selection. Added the ability to use the RSA PSS algorithm for handshake signatures. Added bnrandinterval() and use it in code needing ranges of random bn values. Added functionality to derive early, handshake, and application secrets as per RFC8446. Added handshake state machine from RFC8446. Removed some ASN.1 related code from libcrypto that had not been used since around 2000. Unexported internal symbols and internalized more record layer structs. Removed SHA224 based handshake signatures from consideration for use in a TLS 1.2 handshake. Portable Improvements Added support for assembly optimizations on 32-bit ARM ELF targets. Added support for assembly optimizations on Mingw-w64 targets. Improved Android compatibility Bug Fixes Improved protection against timing side channels in ECDSA signature

Introducing funlinkat(), an OpenBSD Router with AT&T U-Verse, using NetBSD on a raspberry pi, ZFS encryption is still under development, Rump kernel servers and clients tutorial, Snort on OpenBSD 6.4, and more. Headlines Introducing funlinkat It turns out, every file you have ever deleted on a unix machine was probably susceptible to a race condition One of the first syscalls which was created in Unix-like systems is unlink. In FreeBSD this syscall is number 10 (source) and in Linux, the number is dependent on the architecture but for most of them is also the tenth syscall (source). This indicated that this is one of the primary syscalls. The unlink syscall is very simple and we provide one single path to the file that we want to remove. The “removing file” process itself is very interesting so let’s spend a moment to understand the it. First, by removing the file we are removing a link from the directory to it. In Unix-like systems we can have many links to a single file (hard links). When we remove all links to the file, the file system will mark the blocks used by the file as free (a different file system will behave differently but let’s not jump into a second digression). This is why the process is called unlinking and not “removing file”. While we unlink the file two or three things will happen: We will remove an entry in the directory with the filename. We will decrease a file reference count (in inode). If links go to zero - the file will be removed from the disk (again this doesn't mean that the blocks from the disk will be filled with zeros, though this may happen depending on the file system and configuration. However, in most cases this means that the file system will mark those blocks to as free and use them to write new data later This mostly means that “removing file” from a directory is an operation on the directory and not on the file (inode) itself. Another interesting subject is what happens if our system will perform only first or second step from the list. This depends on the file system and this is also something we will leave for another time. The problem with the unlink and even unlinkat function is that we don’t have any guarantee of which file we really are unlinking. When you delete a file using its name, you have no guarantee that someone has not already deleted the file, or renamed it, and created a new file with the name you are about to delete. We have some stats about the file that we want to unlink. We performed some tests. In the same time another process removed our file and recreated it. When we finally try to remove our file it is no longer the same file. It’s a classic race condition. Many programs will perform checks before trying to remove a file, to make sure it is the correct file, that you have the correct permissions etc. However this exposes the ‘Time-of-Check / Time-of-Use’ class of bugs. I check if the file I am about to remove is the one I created yesterday, it is, so I call unlink() on it. However, between when I checked the date on the file, and when I call unlink, I, some program I am running, might have updated the file. Or a malicious user might have put some other file at that name, so I would be the one who deleted it. In Unix-like operating systems we can get a handle for our file called file - a descriptor. File descriptors guarantee us that all the operations that we will be performing on it are done on the same file (inode). Even if someone was to unlink a number of directories entries, the operating system will not free the structures behind the file descriptor, and we can detect the file that was removed by someone and recreated (or just unlinked). So, for example, we have an alternative functions fstat which allows us to get file status of the given descriptor We already know that the file may have many links on the disk which point to the single inode. What happens when we open the file? Simplifying: kernel creates a memory representation of the inode (the inode itself is stored on the disk) called vnode. This single representation is used by all processes to refer the inode to the disk. If in a process we open the same file (inode) using different names (for example through hard links) all those files will be linked to the single vnode. That means that the pathname is not stored in the kernel. This is basically the reason why we don’t have a funlink function so that instead of the path we are providing just the file descriptor to the file. If we performed the fdunlink syscall, the kernel wouldn’t know which directory entry you would like to remove. Another problem is more architectural: as we discussed earlier unlinking is really an operation on the directory not on the file (inode) itself, so using funlink(fd) may create some confusion because we are not removing the inode corresponding to the file descriptor, we are performing action on the directory which points to the file. After some discussion we decided that the only sensible option

A PI-powered Plan 9 cluster, an SSH tarpit, rdist for when Ansible is too much, falling in love with OpenBSD again, how I created my first FreeBSD port, the Tilde Institute of OpenBSD education and more. Headlines A Pi-Powered Plan 9 Cluster Plan 9 from Bell Labs comes from the same stable as the UNIX operating system, which of course Linux was designed after, and Apple’s OS X runs on top of a certified UNIX operating system. Just like UNIX, Plan 9 was developed as a research O/S — a vehicle for trying out new concepts — with it building on key UNIX principles and taking the idea of devices are just files even further. In this post, we take a quick look at the Plan 9 O/S and some of the notable features, before moving on to the construction of a self-contained 4-node Raspberry Pi cluster that will provide a compact platform for experimentation. Endlessh: an SSH Tarpit I’m a big fan of tarpits: a network service that intentionally inserts delays in its protocol, slowing down clients by forcing them to wait. This arrests the speed at which a bad actor can attack or probe the host system, and it ties up some of the attacker’s resources that might otherwise be spent attacking another host. When done well, a tarpit imposes more cost on the attacker than the defender. The Internet is a very hostile place, and anyone who’s ever stood up an Internet-facing IPv4 host has witnessed the immediate and continuous attacks against their server. I’ve maintained such a server for nearly six years now, and more than 99% of my incoming traffic has ill intent. One part of my defenses has been tarpits in various forms. News Roundup rdist(1) – when Ansible is too much The post written about rdist(1) on johan.huldtgren.com sparked us to write one as well. It's a great, underappreciated, tool. And we wanted to show how we wrapped doas(1) around it. There are two services in our infrastructure for which we were looking to keep the configuration in sync and to reload the process when the configuration had indeed changed. There is a pair of nsd(8)/unbound(8) hosts and a pair of hosts running relayd(8)/httpd(8) with carp(4) between them. We didn't have a requirement to go full configuration management with tools like Ansible or Salt Stack. And there wasn't any interest in building additional logic on top of rsync or repositories. > Enter rdist(1), rdist is a program to maintain identical copies of files over multiple hosts. It preserves the owner, group, mode, and mtime of files if possible and can update programs that are executing. Falling in love with OpenBSD again I was checking the other day and was appalled at how long it has been since I posted here. I had been working a job during 2018 that had me traveling 3,600 miles by air every week so that is at least a viable excuse. So what is my latest project? I wanted to get something better than the clunky old T500 “freedom laptop” that I could use as my daily driver. Some background here. My first paid gig as a programmer was on SunOS 4 (predecessor to Solaris) and Ultrix (on a DEC MicroVAX). I went from there to a Commodore Amiga (preemptive multitasking in 1985!). I went from there to OS/2 (I know, patron saint of lost causes) and then finally decided to “sell out” and move to Windows as the path of least resistance in the mid 90’s. My wife bought me an iPod literally just as they started working with computers other than Macs and I watched with fascination as Apple made the big gamble and moved away from PowerPC chips to Intel. That was the beginning of the Apple Fan Boi years for me. My gateway drug was a G4 MacMini and I managed somehow to get in on the pre-production, developer build of an Intel-based Mac. I was quite happy on the platform until about three years ago. How I Created My First FreeBSD Port I created my first FreeBSD port recently. I found that FreeBSD didn't have a port for GoCD, which is a continuous integration and continuous deployment (CI/CD) system. This was a great opportunity to learn how to build a FreeBSD port while also contributing back to the community The Tilde Institute of OpenBSD Education Welcome to tilde.institute! This is an OpenBSD machine whose purpose is to provide a space in the tildeverse for experimentation with and education of the OpenBSD operating system. A variety of editors, shells, and compilers are installed to allow for development in a native OpenBSD environment. OpenBSD's httpd(8) is configured with slowcgi(8) as the fastcgi provider and sqlite3 available. This allows users to experiment with web development using compiled CGI in C, aka the BCHS Stack. In addition to php7.0 and mysql (mariadb) by request, this provides an environment where the development of complex web apps is possible. Beastie Bits SoloBSD 19.03-STABLE WireGuard for NetBSD [NetBSD - Removing PF](https://mail-index.netbsd.org/tech-kern/2019/03/29/msg024883.html ) What does the N in nmake stand for? A Map of the Internet from May 1973 NSA-B-Gone : A sketchy hard

This week we have a special episode with a Michael W. Lucas interview about his latest jail book that’s been released. We’re talking all things jails, writing, book sponsoring, the upcoming BSDCan 2019 conference, and more. ###Interview - Michael W. Lucas - [email protected] / @mwlauthor FreeBSD Mastery: Jails BR: Welcome back to the show and congratulations on your latest book. How many books did you have to write before you could start on FreeBSD Mastery: Jails? AJ: How much research did you have to do about jails? BR: The book talks about something called ‘incomplete’ jails. What do you mean by that? AJ: There are a lot of jail management frameworks out there. Why did you chose to write about iocage in the book? BR: How many jails do you run yourself? AJ: Can you tell us a bit about how you handle book sponsorship these days? BR: What other books (fiction and non-fiction) are you currently working on? AJ: Which talks are you looking forward to attend at the upcoming BSDCan conference? BR: How is the BSD user group going? AJ: Anything else you’d like to mention before we release you from our interview jail cell? Send questions, comments, show ideas/topics, or stories you want mentioned on the show to [email protected] Your browser does not support the HTML5 video tag.

FreeBSD Q4 2018 status report, the GhostBSD alternative, the coolest 90s laptop, OpenSSH 8.0 with quantum computing resistant keys exchange, project trident: 18.12-U8 is here, and more. ##Headlines ###AsiaBSDcon 2019 recap Both Allan and I attended AsiaBSDcon 2019 in Tokyo in mid march. After a couple of days of Tokyo sightseeing and tasting the local food, the conference started with tutorials. Benedict gave his tutorial about “BSD-based Systems Monitoring with Icinga2 and OpenSSH”, while Allan ran the FreeBSD developer summit. On the next day, Benedict attended the tutorial “writing (network) tests for FreeBSD” held by Kristof Provost. I learned a lot about Kyua, where tests live and how they are executed. I took some notes, which will likely become an article or chapter in the developers handbook about writing tests. On the third day, Hiroki Sato officially opened the paper session and then people went into individual talks. Benedict attended Adventure in DRMland - Or how to write a FreeBSD ARM64 DRM driver by Emmanuel Vadot powerpc64 architecture support in FreeBSD ports by Piotr Kubaj Managing System Images with ZFS by Allan Jude FreeBSD - Improving block I/O compatibility in bhyve by Sergiu Weisz Security Fantasies and Realities for the BSDs by George V. Neville-Neil ZRouter: Remote update of firmware by Hiroki Mori Improving security of the FreeBSD boot process by Marcin Wojtas Allan attended Adventures in DRMland by Emmanuel Vadot Intel HAXM by Kamil Rytarowski BSD Solutions in Australian NGOs Container Migration on FreeBSD by Yuhei Takagawa Security Fantasies and Realities for the BSDs by George Neville-Neil ZRouter: Remote update of firmware by Hiroki Mori Improving security of the FreeBSD boot process by Marcin Wojtas When not in talks, time was spent in the hallway track and conversations would often continue over dinner. Stay tuned for announcements about where AsiaBSDcon 2020 will be, as the Tokyo Olympics will likely force some changes for next year. Overall, it was nice to see people at the conference again, listen to talks, and enjoy the hospitality of Japan. ###FreeBSD Quarterly Status Report - Fourth Quarter 2018 Since we are still on this island among many in this vast ocean of the Internet, we write this message in a bottle to inform you of the work we have finished and what lies ahead of us. These deeds that we have wrought with our minds and hands, they are for all to partake of - in the hopes that anyone of their free will, will join us in making improvements. In todays message the following by no means complete or ordered set of improvements and additions will be covered: i386 PAE Pagetables for up to 24GB memory support, Continuous Integration efforts, driver updates to ENA and graphics, ARM enhancements such as RochChip, Marvell 8K, and Broadcom support as well as more DTS files, more Capsicum possibilities, as well as pfsync improvements, and many more things that you can read about for yourselves. Additionally, we bring news from some islands further down stream, namely the nosh project, HardenedBSD, ClonOS, and the Polish BSD User-Group. We would, selfishly, encourage those of you who give us the good word to please send in your submissions sooner than just before the deadline, and also encourage anyone willing to share the good word to please read the section on which submissions we’re also interested in having. ###GhostBSD: A Solid Linux-Like Open Source Alternative The subject of this week’s Linux Picks and Pans is a representative of a less well-known computing platform that coexists with Linux as an open source operating system. If you thought that the Linux kernel was the only open source engine for a free OS, think again. BSD (Berkeley Software Distribution) shares many of the same features that make Linux OSes viable alternatives to proprietary computing platforms. GhostBSD is a user-friendly Linux-like desktop operating system based on TrueOS. TrueOS is, in turn, based on FreeBSD’s development branch. TrueOS’ goal is to combine the stability and security of FreeBSD with a preinstalled GNOME, MATE, Xfce, LXDE or Openbox graphical user interface. I stumbled on TrueOS while checking out new desktop environments and features in recent new releases of a few obscure Linux distros. Along the way, I discovered that today’s BSD computing family is not the closed source Unix platform the “BSD” name might suggest. In last week’s Redcore Linux review, I mentioned that the Lumina desktop environment was under development for an upcoming Redcore Linux release. Lumina is being developed primarily for BSD OSes. That led me to circle back to a review I wrote two years ago on Lumina being developed for Linux. GhostBSD is a pleasant discovery. It has nothing to do with being spooky, either. That goes for both the distro and the open source computing family it exposes. Keep reading to find out what piqued my excitement about Linux-like GhostBSD. ##News Roundup ###SPARCbook 3000ST - The co

Storage changing software, what makes Unix special, what you need may be “pipeline +Unix commands”, running a bakery on Emacs and PostgreSQL, the ultimate guide to memorable tech talks, light-weight contexts, and more. ##Headlines ###Tracking a storage issue led to software change Early last year we completed a massive migration that moved our customers’ hosting data off of a legacy datacenter (that we called FR-SD2) onto several new datacenters (that we call FR-SD3, FR-SD5, and FR-SD6) with much more modern, up-to-date infrastructure. This migration required several changes in both the software and hardware we use, including switching the operating system on our storage units to FreeBSD. Currently, we use the NFS protocol to provide storage and export the filesystems on Simple Hosting, our web hosting service, and the FreeBSD kernel includes an NFS server for just this purpose. Problem While migrating virtual disks of Simple Hosting instances from FR-SD2, we noticed high CPU load spikes on the new storage units. ###What Makes Unix Special Ever since Unix burst onto the scene within the early '70s, observers within the pc world have been fast to put in writing it off as a unusual working system designed by and for knowledgeable programmers. Regardless of their proclamations, Unix refuses to die. Means again in 1985, Stewart Cheifet puzzled if Unix would turn out to be the usual working system of the longer term on the PBS present “The Laptop Chronicles,” though MS-DOS was effectively in its heyday. In 2018, it is clear that Unix actually is the usual working system, not on desktop PCs, however on smartphones and tablets. What Makes Unix Special? It is also the usual system for net servers. The actual fact is, hundreds of thousands of individuals all over the world have interacted with Linux and Unix programs daily, most of whom have by no means written a line of code of their lives. So what makes Unix so beloved by programmers and different techie sorts? Let’s check out a few of issues this working system has going for it. (For some background on Unix, try The Historical past of Unix: From Bell Labs to the iPhone.) ##News Roundup ###What you need may be “pipeline +Unix commands” only I came across Taco Bell Programming recently, and think this article is worthy to read for every software engineer. The post mentions a scenario which you may consider to use Hadoop to solve but actually xargs may be a simpler and better choice. This reminds me a similar experience: last year a client wanted me to process a data file which has 5 million records. After some investigations, no novel technologies, a concise awk script (less than 10 lines) worked like a charm! What surprised me more is that awk is just a single-thread program, no nifty concurrency involved. The IT field never lacks “new” technologies: cloud computing, big data, high concurrency, etc. However, the thinkings behind these “fancy” words may date back to the era when Unix arose. Unix command line tools are invaluable treasure. In many cases, picking the right components and using pipeline to glue them can satisfy your requirement perfectly. So spending some time in reviewing Unixcommand line manual instead of chasing state-of-the-art techniques exhaustedly, you may gain more. BTW, if your data set can be disposed by an awk script, it should not be called “big data”. Taco Bell Programming ###Running a bakery on Emacs and PostgreSQL Just over a year ago now, I finally opened the bakery I’d been dreaming of for years. It’s been a big change in my life, from spending all my time sat in front of a computer, to spending most of it making actual stuff. And stuff that makes people happy, at that. It’s been a huge change, but I can’t think of a single job change that’s ever made me as happy as this one. One of the big changes that came with going pro was that suddenly I was having to work out how much stuff I needed to mix to fill the orders I needed. On the face of it, this is really simple, just work out how much dough you need, then work out what quantities to mix to make that much dough. Easy. You can do it with a pencil and paper. Or, in traditional bakers’ fashion, by scrawling with your finger on a floured work bench. And that’s how I coped for a few weeks early on. But I kept making mistakes, which makes for an inconsistent product (bread is very forgiving, you have to work quite hard to make something that isn’t bread, but consistency matters). I needed to automate. ###The Ultimate Guide To Memorable Tech Talks Imagine this. You’re a woman in a male-dominated field. English is not your first language. Even though you’re confident in your engineering work, the thought of public speaking and being recorded for the world to see absolutely terrifies you. That was me, five years ago. Since then, I’ve moved into a successful career in Developer Advocacy and spoken at dozens of technical events in the U.S. and worldwide. I think everyone has the ability to del

FreeBSD on Cavium ThunderX, looking at NetBSD as an OpenBSD user, taking time-stamped notes in vim, OpenBSD 6.5 has been tagged, FreeBSD and NetBSD in GSoC 2019, SecBSD: an UNIX-like OS for Hackers, and more. ##Headlines ###ARM’d and dangerous: FreeBSD on Cavium ThunderX (aarch64) While I don’t remember for how many years I’ve had an interest in CPU architectures that could be an alternative to AMD64, I know pretty well when I started proposing to test 64-bit ARM at work. It was shortly after the disaster named Spectre / Meltdown that I first dug out server-class ARM hardware and asked whether we should get one such server and run some tests with it. While the answer wasn’t a clear “no” it also wasn’t exactly “yes”. I tried again a few times over the course of 2018 and each time I presented some more points why I thought it might be a good thing to test this. But still I wasn’t able to get a positive answer. Finally in January 2019 year I got a definitive answer – and it was “yes, go ahead”! The fact that Amazon had just presented their Graviton ARM Processor may have helped the decision. ###Looking at NetBSD from an OpenBSD user perspective I use to use NetBSD quite a lot. From 2.0 to 6.99. But for some reasons, I stopped using it about 2012, in favor of OpenBSD. Reading on the new 8 release, I wanted to see if all the things I didn’t like on NetBSD were gone. Here is a personal Pros / Cons list. No Troll, hopefully. Just trying to be objective. What I liked (pros) Things I didn’t like (cons) Conclusion So that was it. I didn’t spend more than 30 minutes of it. But I didn’t want to spend more time on it. I did stop using NetBSD because of the need to compile each and every packages ; it was in the early days of pkgin. I also didn’t like the way system maintenance was to be done. OpenBSD’s 6-months release seemed far more easy to manage. I still think NetBSD is a great OS. But I believe you have to spent more time on it than you would have to do with OpenBSD. That said, I’ll keep using my Puffy OS. ##News Roundup ###Using Vim to take time-stamped notes I frequently find myself needing to take time-stamped notes. Specifically, I’ll be in a call, meeting, or interview and need to take notes that show how long it’s been since the meeting started. My first thought was that there’s be a plugin to add time stamps, but a quick search didn’t turn anything up. However, I little digging did turn up the fact that vim has the built-in ability to tell time. This means that writing a bit of vimscript to insert a time stamp is pretty easy. After a bit of fiddling, I came up with something that serves my needs, and I decided it might be useful enough to others to be worth sharing. John Baldwin’s notes on bhyve meetings ###OpenBSD 6.5-beta has been tagged It’s that time of year again; Theo (deraadt@) has just tagged 6.5-beta. A good reminder for us all run an extra test install and see if your favorite port still works as you expect. CVSROOT: /cvs Module name: src Changes by: [email protected] 2019/02/26 15:24:41 Modified files: etc/root : root.mail share/mk : sys.mk sys/conf : newvers.sh sys/sys : ktrace.h param.h usr.bin/signify: signify.1 sys/arch/macppc/stand/tbxidata: bsd.tbxi Log message: crank to 6.5-beta ###The NetBSD Foundation participating in Google Summer of Code 2019 For the 4th year in a row and for the 13th time The NetBSD Foundation will participate in Google Summer of Code 2019! If you are a student and would like to learn more about Google Summer of Code please go to the Google Summer of Code homepage. You can find a list of projects in Google Summer of Code project proposals in the wiki. Do not hesitate to get in touch with us via #netbsd-code IRC channel on Freenode and via NetBSD mailing lists! ###SecBSD: an UNIX-like OS for Hackers SecBSD is an UNIX-like operating system focused on computer security based on OpenBSD. Designed for security testing, hacking and vulnerability assessment, it uses full disk encryption and ProtonVPN + OpenVPN by default. A security BSD enviroment for security researchers, penetration testers, bug hunters and cybersecurity experts. Developed by Dark Intelligence Team for private use and will be public release coming soon. ##Beastie Bits Why OpenBSD Rocks Rich’s sh (POSIX shell) tricks Drinking coffee with AWK Civilisational HTTP Error Codes MidnightBSD Roadmap NetBSD on Nintendo64 From Vimperator to Tridactyl ##Feedback/Questions Russell - BSD Now Question :: ZFS & FreeNAS Alan - Tutorial, install ARM *BSD with no other BSD box pls Johnny - New section to add to the show Send questions, comments, show ideas/topics, or stories you want mentioned on the show to [email protected] Your browser does not support the HTML5 video tag.

A kernel of failure, IPv6 fragmentation vulnerability in OpenBSD’s pf, a guide to the terminal, using a Yubikey for SSH public key authentication, FreeBSD desktop series, and more. ##Headlines ###A Kernel Of Failure - How IBM bet big on the microkernel being the next big thing in operating systems back in the ’90s—and spent billions with little to show for it. Today in Tedium: In the early 1990s, we had no idea where the computer industry was going, what the next generation would look like, or even what the driving factor would be. All the developers back then knew is that the operating systems available in server rooms or on desktop computers simply weren’t good enough, and that the next generation needed to be better—a lot better. This was easier said than done, but this problem for some reason seemed to rack the brains of one company more than any other: IBM. Throughout the decade, the company was associated with more overwrought thinking about operating systems than any other, with little to show for it in the end. The problem? It might have gotten caught up in kernel madness. Today’s Tedium explains IBM’s odd operating system fixation, and the belly flops it created. ###CVE-2019-5597IPv6 fragmentation vulnerability in OpenBSD Packet Filter Packet Filter is OpenBSD’s service for filtering network traffic and performing Network Address Translation. Packet Filter is also capable of normalizing and conditioning TCP/IP traffic, as well as providing bandwidth control and packet prioritization. Packet Filter has been a part of the GENERIC kernel since OpenBSD 5.0.Because other BSD variants import part of OpenBSD code, Packet Filter is also shipped with at least the following distributions that are affected in a lesser extent: FreeBSD, pfSense, OPNSense, Solaris. Note that other distributions may also contain Packet Filter but due to the imported version they might not be vulnerable. This advisory covers the latest OpenBSD’s Packet Filter. For specific details about other distributions, please refer to the advisory of the affected product. Kristof Provost, who maintains the port of pf in FreeBSD added a test for the vulnerability in FreeBSD head. ##News Roundup ###How I’m still not using GUIs in 2019: A guide to the terminal TL;DR: Here are my dotfiles. Use them and have fun. GUIs are bloatware. I’ve said it before. However, rather than just complaining about IDEs I’d like to provide an understandable guide to a much better alternative: the terminal. IDE stands for Integrated Development Environment. This might be an accurate term, but when it comes to a real integrated development environment, the terminal is a lot better. In this post, I’ll walk you through everything you need to start making your terminal a complete development environment: how to edit text efficiently, configure its appearance, run and combine a myriad of programs, and dynamically create, resize and close tabs and windows. Don’t forget rule number one. Whenever in doubt, read the manual. ###Using a Yubikey as smartcard for SSH public key authentication SSH is an awesome tool. Logging into other machines securely is so pervasive to us sysadmins nowadays that few of us think about what’s going on underneath. Even more so once you start using the more advanced features such as the ssh-agent, agent-forwarding and ProxyJump. When doing so, care must be taken in order to not compromise one’s logins or ssh keys. You might have heard of Yubikeys. These are USB authentication devices that support several different modes: they can be used for OTP (One Time Password) authentication, they can store OpenPGP keys, be a 2-factor authentication token and they can act as a SmartCard. In OpenBSD, you can use them for Login (with login_yubikey(8)) with OTP since 2012, and there are many descriptions available(1) how to set this up. ###The 18 Part FreeBSD Desktop Series by Vermaden FreeBSD Desktop – Part 1 – Simplified Boot FreeBSD Desktop – Part 2 – Install (FreeBSD 11) FreeBSD Desktop – Part 2.1 – Install FreeBSD 12 FreeBSD Desktop – Part 3 – X11 Window System FreeBSD Desktop – Part 4 – Key Components – Window Manager FreeBSD Desktop – Part 5 – Key Components – Status Bar FreeBSD Desktop – Part 6 – Key Components – Task Bar FreeBSD Desktop – Part 7 – Key Components – Wallpaper Handling FreeBSD Desktop – Part 8 – Key Components – Application Launcher FreeBSD Desktop – Part 9 – Key Components – Keyboard/Mouse Shortcuts FreeBSD Desktop – Part 10 – Key Components – Locking Solution FreeBSD Desktop – Part 11 – Key Components – Blue Light Spectrum Suppress FreeBSD Desktop – Part 12 – Configuration – Openbox FreeBSD Desktop – Part 13 – Configuration – Dzen2 FreeBSD Desktop – Part 14 – Configuration – Tint2 FreeBSD Desktop – Part 15 – Configuration – Fonts & Frameworks FreeBSD Desktop – Part 16 – Configuration – Pause Any Application FreeBSD Desktop – Part 17 – Automount Removable Media ##Beastie Bits Drist with persistent SSH ARPANET: Celebrating 50 Years Sinc

Software will never fix Spectre-type bugs, a proof that sed is Turing complete, managed jails using Bastille, new version of netdata, using grep with /dev/null, using GMail with mutt, and more. ##Headlines ###Google: Software is never going to be able to fix Spectre-type bugs Spectre is here to stay: An analysis of side-channels and speculative execution Researchers from Google investigating the scope and impact of the Spectre attack have published a paper asserting that Spectre-like vulnerabilities are likely to be a continued feature of processors and, further, that software-based techniques for protecting against them will impose a high performance cost. And whatever the cost, the researchers continue, the software will be inadequate—some Spectre flaws don’t appear to have any effective software-based defense. As such, Spectre is going to be a continued feature of the computing landscape, with no straightforward resolution. The discovery and development of the Meltdown and Spectre attacks was undoubtedly the big security story of 2018. First revealed last January, new variants and related discoveries were made throughout the rest of the year. Both attacks rely on discrepancies between the theoretical architectural behavior of a processor—the documented behavior that programmers depend on and write their programs against—and the real behavior of implementations. Specifically, modern processors all perform speculative execution; they make assumptions about, for example, a value being read from memory or whether an if condition is true or false, and they allow their execution to run ahead based on these assumptions. If the assumptions are correct, the speculated results are kept; if it isn’t, the speculated results are discarded and the processor redoes the calculation. Speculative execution is not an architectural feature of the processor; it’s a feature of implementations, and so it’s supposed to be entirely invisible to running programs. When the processor discards the bad speculation, it should be as if the speculation never even happened. ###A proof that Unix utility sed is Turing complete Many people are surprised when they hear that sed is Turing complete. How come a text filtering program is Turing complete, they wonder. Turns out sed is a tiny assembly language that has a comparison operation, a branching operation and a temporary buffer. These operations make sed Turing complete. I first learned about this from Christophe Blaess. His proof is by construction – he wrote a Turing machine in sed (download turing.sed). As any programming language that can implement a Turing machine is Turing complete we must conclude that sed is also Turing complete. Christophe offers his own introduction to Turing machines and a description of how his sed implementation works in his article Implementation of a Turing Machine as a sed Script. Christophe isn’t the first person to realize that sed is almost a general purpose programming language. People have written tetris, sokoban and many other programs in sed. Take a look at these: Tetris Sokoban (game) Calculator ##News Roundup ###Bastille helps you quickly create and manage FreeBSD Jails. Bastille helps you quickly create and manage FreeBSD Jails. Jails are extremely lightweight containers that provide a full-featured UNIX-like operating system inside. These containers can be used for software development, rapid testing, and secure production Internet services. Bastille provides an interface to create, manage and destroy these secure virtualized environments. Current version: 0.3.20190204-beta. Shell Script Source here: https://github.com/BastilleBSD/bastille/blob/master/usr/local/bin/bastille ###netdata v1.12 released Netdata is distributed, real-time, performance and health monitoring for systems and applications. It is a highly optimized monitoring agent you install on all your systems and containers. Netdata provides unparalleled insights, in real-time, of everything happening on the systems it runs (including web servers, databases, applications), using highly interactive web dashboards. It can run autonomously, without any third party components, or it can be integrated to existing monitoring tool chains (Prometheus, Graphite, OpenTSDB, Kafka, Grafana, etc). Netdata is fast and efficient, designed to permanently run on all systems (physical & virtual servers, containers, IoT devices), without disrupting their core function. Patch release 1.12.1 contains 22 bug fixes and 8 improvements. ###Using grep with /dev/null, an old Unix trick Every so often I will find myself writing a grep invocation like this: find .... -exec grep <something> /dev/null '{}' '+' The peculiar presence of /dev/null here is an old Unix trick that is designed to force grep to always print out file names, even if your find only matches one file, by always insuring that grep has at least two files as arguments. You can wind up wanting to do the same thing with a direct use of grep

Design and Implementation of NetBSD’s rc.d system, first impressions of Project Trident 18.12, PXE booting a FreeBSD disk image, middle mouse button pasting, NetBSD gains hardware accelerated virtualization, and more. ##Headlines ###The Design and Implementation of the NetBSD rc.d system Abstract In this paper I cover the design and implementation of the rc.d system start-up mechanism in NetBSD 1.5, which replaced the monolithic /etc/rc start-up file inherited from 4.4BSD. Topics covered include a history of various UNIX start-up mechanisms (including NetBSD prior to 1.5), design considerations that evolved over six years of discussions, implementation details, an examination of the human issues that occurred during the design and implementation, as well as future directions for the system. Introduction NetBSD recently converted from the traditional 4.4BSD monolithic /etc/rc start-up script to an /etc/rc.d mechanism, where there is a separate script to manage each service or daemon, and these scripts are executed in a specific order at system boot. This paper covers the motivation, design and implementation of the rc.d system; from the history of what NetBSD had before to the system that NetBSD 1.5 shipped with in December 2000, as well as future directions. The changes were contentious and generated some of the liveliest discussions about any feature change ever made in NetBSD. Parts of those discussions will be covered to provide insight into some of the design and implementation decisions. History There is great diversity in the system start-up mechanisms used by various UNIX variants. A few of the more pertinent schemes are detailed below. As NetBSD is derived from 4.4BSD, it follows that a description of the latter’s method is relevant. Solaris’ start-up method is also detailed, as it is the most common System V UNIX variant. ###First impressions of Project Trident 18.12 Project Trident (hereafter referred to as Trident) is a desktop operating system based on TrueOS. Trident takes the rolling base platform of TrueOS, which is in turn based on FreeBSD’s development branch, and combines it with the Lumina desktop environment. +Installing The debut release of Trident is available as a 4.1GB download that can be burned to a disc or transferred to a USB thumb drive. Booting from the Trident media brings up a graphical interface and automatically launches the project’s system installer. Down the left side of the display there are buttons we can click to show hardware information and configuration options. These buttons let us know if our wireless card and video card are compatible with Trident and give us a chance to change our preferred language and keyboard layout. At the bottom of the screen we find buttons that will open a terminal or shutdown the computer. Early impressions Trident boots to a graphical login screen where we can sign into the Lumina desktop or a minimal Fluxbox session. Lumina, by default, uses Fluxbox as its window manager. The Lumina desktop places its panel along the bottom of the screen and an application menu sits in the bottom-left corner. On the desktop we find icons for opening the software manager, launching the Falkon web browser, running the VLC media player, opening the Control Panel and adjusting the Lumina theme. The application menu has an unusual and compact layout. The menu shows just a search box and buttons for browsing applications, opening a file manager, accessing desktop settings and signing out. To see what applications are available we can click the Browse Applications entry, which opens a window in the menu where we can scroll through installed programs. This is a bit awkward since the display window is small and only shows a few items at a time. Early on I found it is possible to swap out the default “Start menu” with an alternative “Application menu” through the Panels configuration tool. This alternative menu offers a classic tree-style application menu. I found the latter menu easier to navigate as it expands to show all the applications in a selected category. Conclusions I have a lot of mixed feelings and impressions when it comes to Trident. On the one hand, the operating system has some great technology under the hook. It has cutting edge packages from the FreeBSD ecosystem, we have easy access to ZFS, boot environments, and lots of open source packages. Hardware support, at least on my physical workstation, was solid and the Lumina desktop is flexible. ##News Roundup ###PXE booting of a FreeBSD disk image I had to set up a regression and network performance lab. This lab will be managed by a Jenkins, but the first step is to understand how to boot a FreeBSD disk by PXE. This article explains a simple way of doing it. For information, all these steps were done using 2 PC Engines APU2 (upgraded with latest BIOS for iPXE support), so it’s a headless (serial port only, this can be IPMI SoL with different hardware) . THE BIG PICTURE Before explaining all ste

Adding glue to a desktop environment, flashing the BIOS on a PC Engine, revive a Cisco IDS into a capable OpenBSD computer, An OpenBSD WindowMaker desktop, RealTime data compression, the love for pipes, and more. ##Headlines ###Adding Glue To a Desktop Environment In this article we will put some light on a lot of tools used in the world of Unix desktop environment customization, particularly regarding wmctrl, wmutils, xev, xtruss, xwininfo, xprop, xdotools, xdo, sxhkd, xbindkeys, speckeysd, xchainkeys, alttab, triggerhappy, gTile, gidmgr, keynav, and more. If those don’t make sense then this article will help. Let’s hope this can open your mind to new possibilities. With that in mind we can wonder if what’s actually needed from a window manager, presentation and operation, can be split up and complemented with other tools. We can also start thinking laterally, the communication and interaction between the different components of the environment. We have the freedom to do so because the X protocol is transparent and components usually implement many standards for interfacing between windows. It’s like gluing parts together to create a desktop environment. The tools we’ll talk about fall into one of those categories: Debugging Window manipulation Simulation of interaction Extended manipulation Hotkey daemon Layout manager ###Flashing the BIOS on the PC Engines APU4c4 I absolutely love the PC Engines APU devices. I use them for testing HardenedBSD experimental features in more constrained 64-bit environments and firewalls. Their USB and mSATA ports have a few quirks, and I bumped up against a major quirk that required flashing a different BIOS as a workaround. This article details the hacky way in which I went about doing that. What prompted this article is that something in either the CAM or GEOM layer in FreeBSD 11.2 caused the mSATA to hang, preventing file writes. OPNsense 18.7 uses FreeBSD 11.1 whereas the recently-released OPNsense 19.1 uses HardenedBSD 11.2 (based on FreeBSD 11.2). I reached out to PC Engines directly, and they let me know that the issue is a known BIOS issue. Flashing the “legacy” BIOS series would provide me with a working system. It also just so happens that a new “legacy” BIOS version was just released which turns on ECC mode for the RAM. So, I get a working OPNsense install AND ECC RAM! I’ll have one bird for dinner, the other for dessert. Though I’m using an APU4, these instructions should work for the other APU devices. The BIOS ROM download URLs should be changed to reflect the device you’re targeting along with the BIOS version you wish to deploy. SPECIAL NOTE: There be dragons! I’m primarily writing this article to document the procedure for my own purposes. My memory tends to be pretty faulty these days. So, if something goes wrong, please do not hold me responsible. You’re the one at the keyboard. ;) VERY SPECIAL NOTE: We’ll use the mSATA drive for swap space, just in case. Should the swap space be used, it will destroy whatever is on the disk. ##News Roundup ###Revive a Cisco IDS into a capable OpenBSD computer! Even though Cisco equipment is very capable, it tends to become End-of-Life before you can say “planned obsolescence”. Websites become bigger, bandwidths increase, and as a side effect of those “improvements”, routers, firewalls, and in this case, intrusion prevention systems get old quicker and quicker. Apparently, this was also the case for the Cisco IDS-4215 Intrusion Detection Sensor that I was given a few months ago. I’m not too proud to admit that at first, I didn’t care about the machine itself, but rather about the add-on PCI network card with 4 Fast Ethernet interfaces. The sensor has obviously seen better days, as it had a broken front panel and needed some cleaning, but upon a closer inspection under the hood (which is held closed by the 4 screws on top), this IDS consists of an embedded Celeron PC with two onboard Ethernet cards, a 2.5″ IDE hard disk, a CF card, and 2 PCI expansion slots (more on them later). Oh, and don’t forget the nasty server-grade fan, which pushed very little air for the noise it was making. ###An OpenBSD desktop using WindowMaker Since I started using *N?X, I’ve regularly used WindowMaker. I’ve always liked the look and feel, the dock system and the dockapps. It may look a bit oldish nowadays. And that’s enough to try to change this. So here it is, a 2019 flavored WindowMaker Desktop, running on OpenBSD 6.4/amd64. This configuration uses the Nord color-scheme, the Adapta-Nokto-Eta GTK theme and the Moblin Unofficial Icons icon set. I did remove applications icons. I just don’t need them on the bottom of the screen as I heavily use “F11” to pop-up the windows list. To be able to do that and keep the dockapps, I tweaked my ~/GNUstep/Defaults/WMWindowAttributes and created a ~/GNUstep/Library/WindowMaker/Themes/Nord.themed/style. And here it is, the NeXT OpenBSD Desktop! ###RealTime Data Compression In a previous episode, we’ve seen

Strategic thinking to keep FreeBSD relevant, reflecting on the soul of a new machine, 10GbE Benchmarks On Nine Linux Distros and FreeBSD, NetBSD integrating LLVM sanitizers in base, FreeNAS 11.2 distrowatch review, and more. ##Headlines ###Strategic thinking, or what I think what we need to do to keep FreeBSD relevant Since I participate in the FreeBSD project there are from time to time some voices which say FreeBSD is dead, Linux is the way to go. Most of the time those voices are trolls, or people which do not really know what FreeBSD has to offer. Sometimes those voices wear blinders, they only see their own little world (were Linux just works fine) and do not see the big picture (like e.g. competition stimulates business, …) or even dare to look what FreeBSD has to offer. Sometimes those voices raise a valid concern, and it is up to the FreeBSD project to filter out what would be beneficial. Recently there were some mails on the FreeBSD lists in the sense of “What about going into direction X?”. Some people just had the opinion that we should stay where we are. In my opinion this is similarly bad to blindly saying FreeBSD is dead and following the masses. It would mean stagnation. We should not hold people back in exploring new / different directions. Someone wants to write a kernel module in (a subset of) C++ or in Rust… well, go ahead, give it a try, we can put it into the Ports Collection and let people get experience with it. This discussion on the mailinglists also triggered some kind of “where do we see us in the next years” / strategic thinking reflection. What I present here, is my very own opinion about things we in the FreeBSD project should look at, to stay relevant in the long term. To be able to put that into scope, I need to clarify what “relevant” means in this case. FreeBSD is currently used by companies like Netflix, NetApp, Cisco, Juniper, and many others as a base for products or services. It is also used by end‐users as a work‐horse (e.g. mailservers, webservers, …). Staying relevant means in this context, to provide something which the user base is interested in to use and which makes it more easy / fast for the user base to deliver whatever they want or need to deliver than with another kind of system. And this in terms of time to market of a solution (time to deliver a service like a web‐/mail‐/whatever‐server or product), and in terms of performance (which not only means speed, but also security and reliability and …) of the solution. I have categorized the list of items I think are important into (new) code/features, docs, polishing and project infrastructure. Links in the following usually point to documentation/HOWTOs/experiences for/with FreeBSD, and not to the canonical entry points of the projects or technologies. In a few cases the links point to an explanation in the wikipedia or to the website of the topic in question. ###Reflecting on The Soul of a New Machine Long ago as an undergraduate, I found myself back home on a break from school, bored and with eyes wandering idly across a family bookshelf. At school, I had started to find a calling in computing systems, and now in the den, an old book suddenly caught my eye: Tracy Kidder’s The Soul of a New Machine. Taking it off the shelf, the book grabbed me from its first descriptions of Tom West, captivating me with the epic tale of the development of the Eagle at Data General. I — like so many before and after me — found the book to be life changing: by telling the stories of the people behind the machine, the book showed the creative passion among engineers that might otherwise appear anodyne, inspiring me to chart a course that might one day allow me to make a similar mark. Since reading it over two decades ago, I have recommended The Soul of a Machine at essentially every opportunity, believing that it is a part of computing’s literary foundation — that it should be considered our Odyssey. Recently, I suggested it as beach reading to Jess Frazelle, and apparently with perfect timing: when I saw the book at the top of her vacation pile, I knew a fuse had been lit. I was delighted (though not at all surprised) to see Jess livetweet her admiration of the book, starting with the compelling prose, the lucid technical explanations and the visceral anecdotes — but then moving on to the deeper technical inspiration she found in the book. And as she reached the book’s crescendo, Jess felt its full power, causing her to reflect on the nature of engineering motivation. Excited to see the effect of the book on Jess, I experienced a kind of reflected recommendation: I was inspired to (re-)read my own recommendation! Shortly after I started reading, I began to realize that (contrary to what I had been telling myself over the years!) I had not re-read the book in full since that first reading so many years ago. Rather, over the years I had merely revisited those sections that I remembered fondly. On the one hand, these sections are s

We recap FOSDEM 2019, FreeBSD Foundation January update, OPNsense 19.1 released, the hardware-assisted virtualization challenge, ZFS and GPL terror, ClonOS 19.01-RELEASE, and more. Headlines FOSDEM 2019 Recap Allan and I were at FOSDEM 2019 in Brussels, Belgium over the weekend. On the Friday before, we held a FreeBSD Devsummit in a hotel conference room, with 25 people attending. We talked about various topics of interest to the project. You can find the notes on the wiki page. Saturday was the first day of FOSDEM. The FreeBSD Project had a table next to the Illumos Project again. A lot of people visited our table, asked questions, or just said “Hi, I watch BSDNow.tv every week”. We handed out a lot of stickers, pens, swag, and flyers. There was also a full day BSD devroom, with a variety of talks that were well attended. In the main conference track, Allan held a talk explaining how the ZFS ARC works. A lot of people attended the talk and had more questions afterwards. Another well attended talk was by Jonathan Looney about Netflix and FreeBSD. Sunday was another day in the same format, but no bsd devroom. A lot of people visited our table, developers and users alike. A lot of meeting and greeting went on. Overall, FOSDEM was a great success with FreeBSD showing a lot of presence. Thanks to all the people who attended and talked to us. Special thanks to the people who helped out at the FreeBSD table and Rodrigo Osorio for running the BSD devroom again. FreeBSD Foundation Update, January 2019 Dear FreeBSD Community Member, Happy New Year! It’s always exciting starting the new year with ambitious plans to support FreeBSD in new and existing areas. We achieved our fundraising goal for 2018, so we plan on funding a lot of work this year! Though it’s the new year, this newsletter highlights some of the work we accomplished in December. We also put together a list of technologies and features we are considering supporting, and are looking for feedback on what users want to help inform our 2019 development plans. Our advocacy and education efforts are in full swing as we prepare for upcoming conferences including FOSDEM, SANOG33, and SCaLE. Finally, we created a year-end video to talk about the work we did in 2018. That in itself was an endeavor, so please take a few minutes to watch it! We’re working on improving the methods we use to inform the community on the work we are doing to support the Project, and are always open to feedback. Now, sit back, grab a refreshing beverage, and enjoy our newsletter! Happy reading!! Deb OPNsense 19.1 released For more than four years now, OPNsense is driving innovation through modularising and hardening the open source firewall, with simple and reliable firmware upgrades, multi-language support, HardenedBSD security, fast adoption of upstream software updates as well as clear and stable 2-Clause BSD licensing. The 19.1 release, nicknamed “Inspiring Iguana”, consists of a total of 620 individual changes since 18.7 came out 6 months ago, spread out over 12 intermediate releases including the recent release candidates. That is the average of 2 stable releases per month, security updates and important bug fixes included! If we had to pick a few highlights it would be: The firewall alias API is finally in place. The migration to HardenedBSD 11.2 has been completed. 2FA now works with a remote LDAP / local TOTP combination. And the OpenVPN client export was rewritten for full API support as well. These are the most prominent changes since version 18.7: fully functional firewall alias API PIE firewall shaper support firewall NAT rule logging support 2FA via LDAP-TOTP combination WPAD / PAC and parent proxy support in the web proxy P12 certificate export with custom passwords Dpinger is now the default gateway monitor ET Pro Telemetry edition plugin[2] extended IPv6 DUID support Dnsmasq DNSSEC support OpenVPN client export API Realtek NIC driver version 1.95 HardenedBSD 11.2, LibreSSL 2.7 Unbound 1.8, Suricata 4.1 Phalcon 3.4, Perl 5.28 firmware health check extended to cover all OS files, HTTPS mirror default updates are browser cache-safe regarding CSS and JavaScript assets collapsible side bar menu in the default theme language updates for Chinese, Czech, French, German, Japanese, Portuguese and Russian API backup export, Bind, Hardware widget, Nginx, Ntopng, VnStat and Dnscrypt-proxy plugins Here are the full changes against version 19.1-RC2: ipsec: add firewall interface as soon as phase 1 is enabled ipsec: phase 1 selection GUI JavaScript compatibility fix monit: widget improvements and bug fix (contributed by Frank Brendel) ui: fix regression in single host or network subnet select in static pages plugins: os-frr 1.7 updates OSFP outbound rules (contributed by Fabian Franz) plugins: os-telegraf 1.7.4 fixes packet filter input plugins: os-theme-rebellion 1.8.2 adds image colour invert plugins: os-vnstat 1.1[3] plugins: os-zabbix-agent now uses Zabbix version 4.0 src: revert mm

We’re at FOSDEM 2019 this week having fun. We’d never leave you in a lurch, so we have recorded an interview with Niclas Zeising of the FreeBSD graphics team for you. Enjoy. ##Interview - Niclas Zeising - [email protected] / @niclaszeising Interview topic: FreeBSD Graphics Stack BR: Welcome Niclas. Since this is your first time on BSDNow, can you tell us a bit about yourself and how you started with Unix/BSD? AJ: What made you start working in the FreeBSD graphics stack? BR: What is the current status with the FreeBSD graphics stack? AJ: What challenges do you face in the FreeBSD graphics stack? BR: How many people are working in the graphics team and what kind of help do you need there? AJ: You’re also involved in FreeBSD ports and held a poudriere tutorial at last years EuroBSDcon. What kind of feedback did you get and will you give that tutorial again? BR: You’ve been organizing the Stockholm BSD user group meeting. Can you tell us a bit about that, what’s involved, how is it structured? AJ: What conferences do you go to where people could talk to you? BR: Is there anything else you’d like to mention before we let you go? ##Feedback/Questions Casey - TrueOS Troels - zfs send vs zfs send -R matclarke - Orphaned packages Send questions, comments, show ideas/topics, or stories you want mentioned on the show to [email protected]

Project Trident 18.12 released, Spotifyd on NetBSD, OPNsense 18.7.10 is available, Ultra EPYC AMD Powered Sun Ultra 24 Workstation, OpenRsync, LLD porting to NetBSD, and more. ##Headlines ###AsiaBSDCon 2019 Call for Papers You have until Jan 30th to submit Full paper requirement is relaxed a bit this year (this year ONLY!) due to the short submission window. You don’t need all 10-12 pages, but it is still preferred. Send a message to [email protected] with your proposal. Could be either for a talk or a tutorial. Two days of tutorials/devsummit and two days of conference during Sakura season in Tokyo, Japan The conference is also looking for sponsors If accepted, flight and hotel is paid for by the conference ###Project Trident 18.12 Released Twitter account if you want to keep up on project news Screenshots Project Trident Community Telegram Channel DistroWatch Page LinuxActionNews Review RoboNuggie’s in depth review ###Building Spotifyd on NetBSD These are the steps I went through to build and run Spotifyd (this commit at the time of writing) on NetBSD AMD64. It’s a Spotify Connect client so it means I still need to control Spotify from another device (typically my phone), but the audio is played through my desktop… which is where my speakers and headphones are plugged in - it means I don’t have to unplug stuff and re-plug into my phone, work laptop, etc. This is 100% a “good enough for now solution” for me; I have had a quick play with the Go based microcontroller from spotcontrol and that allows a completely NetBSD only experience (although it is just an example application so doesn’t provide many features - great as a basis to build on though). ##News Roundup ###OPNsense 18.7.10 released 2019 means 19.1 is almost here. In the meantime accept this small incremental update with goodies such as Suricata 4.1, custom passwords for P12 certificate export as well as fresh fixes in the FreeBSD base. A lot of cleanups went into this update to make sure there will be a smooth transition to 19.1-RC for you early birds. We expect RC1 in 1-2 weeks and the final 19.1 on January 29. ###Introducing the Ultra EPYC AMD Powered Sun Ultra 24 Workstation A few weeks ago, I got an itch to build a workstation with AMD EPYC. There are a few constraints. First, I needed a higher-clock part. Second, I knew the whole build would be focused more on being an ultra high-end workstation rather than simply utilizing gaming components. With that, I decided it was time to hit on a bit of nostalgia for our readers. Mainly, I wanted to do an homage to Sun Microsystems. Sun made the server gear that the industry ran on for years, and as a fun fact, if you go behind the 1 Hacker Way sign at Facebook’s campus, they left the Sun Microsystems logo. Seeing that made me wonder if we could do an ultimate AMD EPYC build in a Sun Microsystems workstation. ###OpenRsync This is a clean-room implementation of rsync with a BSD (ISC) license. It is designed to be compatible with a modern rsync (3.1.3 is used for testing). It currently compiles and runs only on OpenBSD. This project is still very new and very fast-moving. It’s not ready for wide-spread testing. Or even narrow-spread beyond getting all of the bits to work. It’s not ready for strong attention. Or really any attention but by careful programming. Many have asked about portability. We’re just not there yet, folks. But don’t worry, the system is easily portable. The hard part for porters is matching OpenBSD’s pledge and unveil. ###The first report on LLD porting LLD is the link editor (linker) component of Clang toolchain. Its main advantage over GNU ld is much lower memory footprint, and linking speed. It is of specific interest to me since currently 8 GiB of memory are insufficient to link LLVM statically (which is the upstream default). The first goal of LLD porting is to ensure that LLD can produce working NetBSD executables, and be used to build LLVM itself. Then, it is desirable to look into trying to build additional NetBSD components, and eventually into replacing /usr/bin/ld entirely with lld. In this report, I would like to shortly summarize the issues I have found so far trying to use LLD on NetBSD. ###Ring in the new It’s the second week of 2019 already, which means I’m curious what Nate is going to do with his series This week in usability … reset the numbering from week 1? That series is a great read, to keep up with all the little things that change in KDE source each week — aside from the release notes. For the big ticket items of KDE on FreeBSD, you should read this blog instead. In ports this week (mostly KDE, some unrelated): KDE Plasma has been updated to the latest release, 5.14.5. KDE Applications 18.12.1 were released today, so we’re right on top of them. Marble was fixed for FreeBSD-running-on-Power9. Musescore caught up on 18 months of releases. Phonon updated to 4.10.1, along with its backends. And in development, Qt WebEngine 5.12 has been prepared in the inco

SCP client vulnerabilities, BSDs vs Linux benchmarks on a Tyan EPYC Server, fame for the Unix inventors, Die IPv4, GhostBSD 18.12 released, Unix in pictures, and more. ##Headlines ###scp client multiple vulnerabilities Overview SCP clients from multiple vendors are susceptible to a malicious scp server performing unauthorized changes to target directory and/or client output manipulation. Description Many scp clients fail to verify if the objects returned by the scp server match those it asked for. This issue dates back to 1983 and rcp, on which scp is based. A separate flaw in the client allows the target directory attributes to be changed arbitrarily. Finally, two vulnerabilities in clients may allow server to spoof the client output. Impact Malicious scp server can write arbitrary files to scp target directory, change the target directory permissions and to spoof the client output. Details The discovered vulnerabilities, described in more detail below, enables the attack described here in brief. The attacker controlled server or Man-in-the-Middle(*) attack drops .bash_aliases file to victim’s home directory when the victim performs scp operation from the server. The transfer of extra files is hidden by sending ANSI control sequences via stderr. For example: user@local:~$ scp user@remote:readme.txt . readme.txt 100% 494 1.6KB/s 00:00 user@local:~$ Once the victim launches a new shell, the malicious commands in .bash_aliases get executed. *) Man-in-the-Middle attack does require the victim to accept the wrong host fingerprint. ###FreeBSD 12.0 vs. DragonFlyBSD 5.4 vs. TrueOS 18.12 vs. Linux On A Tyan EPYC Server Last month when running FreeBSD 12.0 benchmarks on a 2P EPYC server I wasn’t able to run any side-by-side benchmarks with the new DragonFlyBSD 5.4 as this BSD was crashing during the boot process on that board. But fortunately on another AMD EPYC server available, the EPYC 1P TYAN Transport SX TN70A-B8026, DragonFlyBSD 5.4.1 runs fine. So for this first round of BSD benchmarking in 2019 are tests of FreeBSD 11.2, FreeBSD 12.0, DragonFlyBSD 5.4.1, the new TrueOS 18.12, and a few Linux distributions (CentOS 7, Ubuntu 18.04.1 LTS, and Clear Linux) on this EPYC 7601 server in a variety of workloads. DragonFlyBSD 5.4.1 ran fine on this Tyan server and could boot fine unlike the issue encountered on the Dell PowerEdge R7425 for this particular BSD. But on the Tyan server, DragonFlyBSD 5.2.2 wouldn’t boot so only this latest DragonFlyBSD release series was used as part of the comparison. A summary of the operating systems tested for this EPYC 7601 OS benchmark comparison included: DragonFlyBSD 5.4.1 - The latest release of Matthew Dillon’s operating system while using the HAMMER2 file-system and GCC 8.1 compiler that is now the default system compiler for this BSD. FreeBSD 11.2 - The previous stable release of FreeBSD. Installed with a ZFS file-system. FreeBSD 12.0 - The latest stable release of FreeBSD and installed with its ZFS option. TrueOS 18.12 - The latest release of the iX systems’ FreeBSD derivative. TrueOS 18.12 is based on FreeBSD 13.0-CURRENT and uses ZFS by default and was using the Clang 7.0.1 compiler compared to Clang 6.0.1 on FreeBSD 12.0. CentOS Linux 7 - The latest EL7 operating system performance. Ubuntu 18.04.1 LTS - The latest Ubuntu Long Term Support release. Clear Linux 27120 - The latest rolling release as of testing out of Intel’s Open-Source Technology Center. Clear Linux often reflects as close to the gold standard for performance as possible with its insanely tuned software stack for offering optimal performance on x86_64 performance for generally showing best what the hardware is capable of. Throughout all of this testing, the Tyan 2U server was kept to its same configuration of an AMD EPYC 7601 (32 cores / 64 threads) at stock speeds, 8 x 16GB DDR4-2666 ECC memory, and 280GB Intel Optane 900p SSD benchmarks. ##News Roundup ###National Inventors Hall of Fame honors creators of Unix Dennis Ritchie (Posthumous) and Ken Thompson: UNIX Operating System Thompson and Ritchie’s creation of the UNIX operating system and the C programming language were pivotal developments in the progress of computer science. Today, 50 years after its beginnings, UNIX and UNIX-like systems continue to run machinery from supercomputers to smartphones. The UNIX operating system remains the basis of much of the world’s computing infrastructure, and C language – written to simplify the development of UNIX – is one of the most widely used languages today. ###Die IPV4, Die Imagine, it is 2019. Easy, ha? Imagine, it is 2019 and you want to turn off IPv4. Like, off off. Really off. Not disabling IPv6, but disabling IPv4. Two steps back You might be coming here wondering, why would anybody want to do what we are asking to be done. Well, it is dead simple: We are running data centers (like Data Center Light) with a lot of IPv6 only equipment. There simply is no need for IPv4. So why would we want to have it enabl

A EULA in FOSS clothing, NetBSD with more LLVM support, Thoughts on FreeBSD 12.0, FreeBSD Performance against Windows and Linux on Xeon, Microsoft shipping NetBSD, and more. Headlines A EULA in FOSS clothing? There was a tremendous amount of reaction to and discussion about my blog entry on the midlife crisis in open source. As part of this discussion on HN, Jay Kreps of Confluent took the time to write a detailed response — which he shortly thereafter elevated into a blog entry. Let me be clear that I hold Jay in high regard, as both a software engineer and an entrepreneur — and I appreciate the time he took to write a thoughtful response. That said, there are aspects of his response that I found troubling enough to closely re-read the Confluent Community License — and that in turn has led me to a deeply disturbing realization about what is potentially going on here. To GitHub: Assuming that this is in fact a EULA, I think it is perilous to allow EULAs to sit in public repositories. It’s one thing to have one click through to accept a license (though again, that itself is dubious), but to say that a git clone is an implicit acceptance of a contract that happens to be sitting somewhere in the repository beggars belief. With efforts like choosealicense.com, GitHub has been a model in guiding projects with respect to licensing; it would be helpful for GitHub’s counsel to weigh in on their view of this new strain of source-available proprietary software and the degree to which it comes into conflict with GitHub’s own terms of service. To foundations concerned with software liberties, including the Apache Foundation, the Linux Foundation, the Free Software Foundation, the Electronic Frontier Foundation, the Open Source Initiative, and the Software Freedom Conservancy: the open source community needs your legal review on this! I don’t think I’m being too alarmist when I say that this is potentially a dangerous new precedent being set; it would be very helpful to have your lawyers offer their perspectives on this, even if they disagree with one another. We seem to be in some terrible new era of frankenlicenses, where the worst of proprietary licenses are bolted on to the goodwill created by open source licenses; we need your legal voices before these creatures destroy the village! NetBSD and LLVM NetBSD entering 2019 with more complete LLVM support I’m recently helping the NetBSD developers to improve the support for this operating system in various LLVM components. As you can read in my previous report, I’ve been focusing on fixing build and test failures for the purpose of improving the buildbot coverage. Previously, I’ve resolved test failures in LLVM, Clang, LLD, libunwind, openmp and partially libc++. During the remainder of the month, I’ve been working on the remaining libc++ test failures, improving the NetBSD clang driver and helping Kamil Rytarowski with compiler-rt. The process of upstreaming support to LLVM sanitizers has been finalized I’ve finished the process of upstreaming patches to LLVM sanitizers (almost 2000LOC of local code) and submitted to upstream new improvements for the NetBSD support. Today out of the box (in unpatched version) we have support for a variety of compiler-rt LLVM features: ASan (finds unauthorized memory access), UBSan (finds unspecified code semantics), TSan (finds threading bugs), MSan (finds uninitialized memory use), SafeStack (double stack hardening), Profile (code coverage), XRay (dynamic code tracing); while other ones such as Scudo (hardened allocator) or DFSan (generic data flow sanitizer) are not far away from completeness. The NetBSD support is no longer visibly lacking behind Linux in sanitizers, although there are still failing tests on NetBSD that are not observed on Linux. On the other hand there are features working on NetBSD that are not functional on Linux, like sanitizing programs during early initialization process of OS (this is caused by /proc dependency on Linux that is mounted by startup programs, while NetBSD relies on sysctl(3) interfaces that is always available). News Roundup Thoughts on FreeBSD 12.0 Playing with FreeBSD with past week I don’t feel as though there were any big surprises or changes in this release compared to FreeBSD 11. In typical FreeBSD fashion, progress tends to be evolutionary rather than revolutionary, and this release feels like a polished and improved incremental step forward. I like that the installer handles both UFS and ZFS guided partitioning now and in a friendly manner. In the past I had trouble getting FreeBSD’s boot menu to work with boot environments, but that has been fixed for this release. I like the security options in the installer too. These are not new, but I think worth mentioning. FreeBSD, unlike most Linux distributions, offers several low-level security options (like hiding other users’ processes and randomizing PIDs) and I like having these presented at install time. It’s harder for people to attack wh

The future of ZFS in FreeBSD, we pick highlights from the FreeBSD quarterly status report, flying with the raven, modern KDE on FreeBSD, many ways to launch FreeBSD in EC2, GOG installers on NetBSD, and more. Headlines The future of ZFS in FreeBSD The sources for FreeBSD’s ZFS support are currently taken directly from Illumos with local ifdefs to support the peculiarities of FreeBSD where the Solaris Portability Layer (SPL) shims fall short. FreeBSD has regularly pulled changes from Illumos and tried to push back any bug fixes and new features done in the context of FreeBSD. In the past few years the vast majority of new development in ZFS has taken place in DelphixOS and zfsonlinux (ZoL). Earlier this year Delphix announced that they will be moving to ZoL: https://www.delphix.com/blog/kickoff-future-eko-2018 This shift means that there will be little to no net new development of Illumos. While working through the git history of ZoL I have also discovered that many races and locking bugs have been fixed in ZoL and never made it back to Illumos and thus FreeBSD. This state of affairs has led to a general agreement among the stakeholders that I have spoken to that it makes sense to rebase FreeBSD’s ZFS on ZoL. Brian Behlendorf has graciously encouraged me to add FreeBSD support directly to ZoL https://github.com/zfsonfreebsd/ZoF so that we might all have a single shared code base. A port for ZoF can be found at https://github.com/miwi-fbsd/zof-port Before it can be committed some additional functionality needs to be added to the FreeBSD opencrypto framework. These can be found at https://reviews.freebsd.org/D18520 This port will provide FreeBSD users with multi modifier protection, project quotas, encrypted datasets, allocation classes, vectorized raidz, vectorized checksums, and various command line improvements. FreeBSD Quarterly Status Update With FreeBSD having gone all the way to 12, it is perhaps useful to take a look back at all the things that have been accomplished, in terms of many visible changes, as well as all the things that happen behind the scenes to ensure that FreeBSD continues to offer an alternative in both design, implementation, and execution. The things you can look forward to reading about are too numerous to summarize, but cover just about everything from finalizing releases, administrative work, optimizations and depessimizations, features added and fixed, and many areas of improvement that might just surprise you a little. Please have a cup of coffee, tea, hot cocoa, or other beverage of choice, and enjoy this culmulative set of reports covering everything that’s been done since October, 2017. —Daniel Ebdrup News Roundup One year of flying with the Raven: Ready for the Desktop? It has been a little over one year now that I’m with the Ravenports project. Time to reflect my involvement, my expectations and hopes. Ravenports Ravenports is a universal packaging framework for *nix operating systems. For the user it provides easy access to binary packages of common software for multiple platforms. It has been the long-lasting champion on Repology’s top 10 repositories regarding package freshness (rarely dropping below 96 percent while all other projects keep below 90!). For the porter it offers a well-designed and elegant means of writing cross-platform buildsheets that allow building the same version of the software with (completely or mostly) the same compile-time configuration on different operating systems or distributions. And for the developer it means a real-world project that’s written in modern Ada (ravenadm) and C (pkg) – as well as some Perl for support scripts and make. Things feel very optimized and fast. Not being a programmer though, I cannot really say anything about the actual code and thus leave it to the interested reader’s judgement. Modern KDE on FreeBSD New stuff in the official FreeBSD repositories! The X11 team has landed a newer version of libinput, opening up the way for KDE Plasma 5.14 in ports. That’s a pretty big update and it may frighten people with a new wallpaper. What this means is that the graphical stack is once again on-par with what Plasma upstream expects, and we can get back to chasing releases as soon as they happen, rather than gnashing our teeth at missing dependencies. The KDE-FreeBSD CI servers are in the process of being upgraded to 12-STABLE, and we’re integrating with the new experimental CI systems as well. This means we are chasing sensibly-modern systems (13-CURRENT is out of scope). The many ways to launch FreeBSD in EC2 Talking to FreeBSD users recently, I became aware that while I’ve created a lot of tools, I haven’t done a very good job of explaining how, and more importantly when to use them. So for all of the EC2-curious FreeBSD users out there: Here are the many ways to launch and configure FreeBSD in EC2 — ranging from the simplest to the most complicated (but most powerful): Launch FreeBSD and SSH in Launch FreeBSD and provide us

We sat down at BSDCan 2018 to interview Kirk McKusick about various topics ranging about the early years of Berkeley Unix, his continuing work on UFS, the governance of FreeBSD, and more. ##Interview - Kirk McKusick - [email protected] 25 years of FreeBSD How Kirk got started in BSD, at the very beginning Predicting the Future How the code and community grew The leadership of the project, and how it changed over time UFS over the years (reading disks from 1982 in 2018) Conferences The rise and fall of Linux The resurgence of FreeBSD We want to extend a big thank you to the entire BSD community for making this show possible, and to all of our viewers for watching and providing the feedback that makes this show successful. We wish you all a happy and prosperous new year, and we’ll see you next week. Send questions, comments, show ideas/topics, or stories you want mentioned on the show to [email protected]