Fazal Majid's low-intensity blog

Sporadic pontification

Fazal

Making the LEDs on the LincStation N2 work in Alpine

TL:DR A daemon to handle the LEDs on the LincStation N2 NAS

I normally run backups on my network on a SmartOS server equipped with 2x14TB mirrored hard drives running ZFS, using rsync and SMB/CIFS as the access protocol (and eventually Kopia). It’s a bit noisy and sluggish, despite having a 10G Ethernet connection, so I recently bought a LincPlus LincStation N2 all-flash server during its Kickstarter to eventually replace it.

The N2 is a very slim device, has 10G (copper) networking, supports 4x M2 SSDs plus 2x SATA 2.5" drives (I suppose they could be spinning rust) and a 128G MMC boot drive so you don’t have to waste data drives on the operating system. The only thing missing, really, is ECC memory.

It is supplied with a trial version of the semi-open Unraid OS, which I suppose is better than the proprietary QNAP QuTS Hero OS or the execrable btrfs-based Synology OS (Synology is beyond the pale because they are going to force their NAS appliance users to use only their own-branded and marked-up hard drives).

I know how to run a Linux system and I don’t need the limitations and hand-holding of a NAS appliance OS, so being able to install my OS of choice, in this case Alpine Linux, was non-negotiable. I managed to, after some fiddling because the device does not honor the BIOS boot settings or even boot off a USB drive until you physically remove the Unraid USB drive plugged into the first M.2 bay using pliers.

Once Alpine was installed, it was fairly smooth sailing, while the 4x4TB Lexar NM790 SSDs are throttled by only one PCIe lane each due to the limitations of the Intel N100, it’s still plenty good enough to saturate the 10Gbps Ethernet connection. My main home server (HP Z2 Mini G4) took this opportunity to die after 6 years of loyal service and the interim replacement I put in place only has 1Gbps, you can feel the difference. I have a Lenovo ThinkStation P3 Ultra on order as a permanent replacement, the only current SFF workstation available with both ECC RAM and 10G Ethernet.

One thing I noticed, however, is that all the status LEDs on the front kept flashing instead of doing so only when a drive or the network sees activity. To make this extra annoying, they are in my peripheral view in my home office. Peripheral vision is of course extremely sensitive to flicker and movement, survival traits when our ancestors in the savanna needed to be on the lookout for predators sneakily approaching. I had to do something about it.

On investigating, it seems they are SMBus devices controlled in closed-source software included on the Unraid flash drive, but also readily available on GitHub. Some reverse-engineering with strace and Hopper Disassembler showed it is written in Go, and constantly spawning subprocesses using the i2c-tools i2cget and i2cset commands, which is quite inefficient.

I first rigged up a simple shell script to just turn off all the LEDs:

#!/bin/sh
# https://gist.github.com/aluevano/ca6431f4f15d8ea62df57e67df7d4c3d

# SATA 1
i2cset -y 11 0x26 0xB0 0x04 # white off
i2cset -y 11 0x26 0xB0 0x08 # red off 
i2cset -y 11 0x26 0x52 0x00 # blinking off

# SATA 2
i2cset -y 11 0x26 0xB0 0x10 # white off
i2cset -y 11 0x26 0xB0 0x20 # red off 
i2cset -y 11 0x26 0x54 0x00 # blinking off

# Network
i2cset -y 11 0x26 0xB0 0x40 # white off
i2cset -y 11 0x26 0xB0 0x80 # red off 
i2cset -y 11 0x26 0x56 0x00 # blinking off

# NVMe 1
i2cset -y 11 0x26 0xB1 0x01 # white off
i2cset -y 11 0x26 0xB1 0x02 # red off 
i2cset -y 11 0x26 0x58 0x00 # blinking off

# NVMe 2
i2cset -y 11 0x26 0xB1 0x04 # white off
i2cset -y 11 0x26 0xB1 0x08 # red off
i2cset -y 11 0x26 0x5A 0x00 # blinking off

# NVMe 3
i2cset -y 11 0x26 0xB1 0x10 # white off
i2cset -y 11 0x26 0xB1 0x20 # red off
i2cset -y 11 0x26 0x5C 0x00 # blinking off

# NVMe 4
i2cset -y 11 0x26 0xB1 0x40 # white off
i2cset -y 11 0x26 0xB1 0x80 # red off
i2cset -y 11 0x26 0x5E 0x00 # blinking off

But this is throwing the baby with the bath water, and losing status indications. I decided to write my own replacement for the LincPlus LED daemon. To save time writing tedious C boilerplate, I asked Claude to write it for me (so am I a systems vibe coder now?). The prompts were:

  1. Using the docs at https://gist.github.com/aluevano/ca6431f4f15d8ea62df57e67df7d4c3d as a guide, write a program in C that uses SMBus/I2C to set LEDs based on disk utilization on sda, sdb, nvme0n1, nvme1n1, nvme2n1, nvme3n1 and network activity
  2. Set MAX_I2C_BUS to a higher value, on my system, the bus found is 11
  3. Use SMBus calls rather than plain I2C whenever possible

Claude did a surprisingly good job. I had to make the following adjustments for it to work properly:

  • Make it use SMBus rather than raw I2C calls to check if the bus supports the LincStation LEDs device 0x26 and avoid false positives (done using prompt 3).
  • Change the code to say utilization is 0% and not 100% if io_time is unchanged.
  • Make the network activity check only eth0, if you are running Docker or LXC, there will be other virtual or bridge interfaces that don’t actually generate outside traffic.

The most impressive thing was how it was able to reverse-engineer the gist and infer a structure to the I2C calls.

All in all, this took about 4 hours’ work including the (non AI assisted) research into how the LEDs work and some attempted reverse-engineering of the LincStation daemon using Hopper Disassembler.

Here is the result: https://github.com/fazalmajid/linstation_leds. I will probably have to make some tweaks as the network activity LED is now solidly on, due to background activity on the network.

Paris travel tips

TL:DR Some quick tips for friends & family travelling to Paris

France is the world’s most popular destination for tourism, and its capital is of course the entry point for most visitors. The Paris region has a population of 9 million, but Paris intra muros (Paris within the walls, the deep historical core city within the Périphérique ring road) is only about 2 million.

Driving within Paris proper is a nightmare, and completely unnecessary given the quality of the regional public transit network. Going to suburbs like Versailles or Saint-Germain en Laye is easy, but going from one suburb to another is harder because the public transport network is radial, you often have to go all the way back to Paris to transfer, although the situation is improving.

The CityMapper app is very useful for figuring out the best transit options to your chosen destination. Since 2005, the entire region has a flat €2.50 charge, or €2 if you limit yourself to buses or the Paris Métro (this does not include the airports, for which there is a surcharge). The easiest way to move around is to use the official Île de France Mobilités app, you can load individual tickets or day/week passes onto your phone if it supports NFC as most do nowadays.

Of course, France is renowned for its food and no visit to Paris is complete without sampling its treats. I’ve made a custom map. David Lebovitz’s blog and book The Good Life in Paris is also an excellent resource, and more up-to-date than anything I can produce from afar.

I won’t cover the major tourist attractions, but here are a few off-the-beaten-path ones:

  • The Musée des Arts & Métiers has a wealth of ancient scientific and technical exhibits, like Foucault’s original pendulum, king Louis XIV’s machines (he had a hobby building mechanical machines), Caselli’s Pantelegraph, a 1860s fax machine, and much much more.
  • The Palais Galliéra is a museum dedicated to fashion. It doesn’t have permanent collection, only rotating special exhibitions, and sometimes workshops as well.
  • The Promenade Plantée is the inspiration for New York’s High Line and other imitators. It’s a raised railway viaduct converted into an urban park, and under its arches the Viaduc des Arts houses arts & crafts shops as well as a number of foodie outlets.
  • The Musée de la Magie is a museum devoted to stage magic, in one of the oldest parts of Paris where you can still see the remains of Philippe-Auguste’s fortifications nearby. It has workshops for children on Wednesday, with some coverage in English, but you would be best asking if one of the teachers is proficient first.
  • The Fragonard perfume company has English-language workshops in their flagship store near the Opéra.
  • The Musée des Plans-Reliefs (English brochure) in the majestic Hôtel des Invalides has a unique collection of 3D scale models of cities and fortifications, built for military planning purposes from Louis XIV onwards.
  • Outside Paris in Élancourt, the France Miniature amusement park has scale models of all of France’s top attractions, plus a few more physical activities like ziplines. You do need to drive to get there, however.
  • The Terrasse de Saint-German-en-Laye and the Parc de Saint-Cloud have outstanding views of Paris.
  • Lines for the Eiffel Tower are often very long. The views from the much less crowded Tour Maine Montparnasse are just as good, and as a bonus you can’t see that modernist carbunkle from itself…

Apple privacy checklist

TL:DR Apple’s claims to being privacy-first are a marketing sham

Apple claims to hold privacy at its core, but it has been an advertising company for at least a decade, and now that smartphone and computer sales are plateauing and new products like the Apple Vision Pro have failed to set the world on fire, Services revenue (an euphemism for the 30% App Store tax on developers and advertising) is critical to maintaining the company’s stock price.

Recent behavior from Apple has confirmed Google or Meta’s take that Apple’s privacy claims are just that, clever marketing to obscure the fact the privacy measures they do have are mainly there to stymie its competitors:

  • Apple forces app developers to ask permission to access the advertising tracking ID (IDFA), but exempts itself from that requirement by a truly Clintonesque redefinition of tracking as “sharing data with other companies, not with ourselves”—one rule for thee but not for me

  • Apple’s notarizarion feature leaks information to Apple on what apps you have installed on your device. What’s worse, this is sent unencrypted so anyone with network access can also grab this info. Apple promised to give a way to disable this misfeature (which also has a noticeable performance impact for developers) but quietly reneged on this.

  • Apple would upload recordings of Siri queries without your consent, and Apple employees and contractors had access to them

  • Apple implemented a CSAM scanning feature, whereby your iPhone would rat you out before the government even asked them to do so. Even though they reversed themselves, they set a precedent authoritarian governments will certainly avail themselves of.

  • When they introduced the Journal app, they gave them wide-ranging access to other apps’ data without consent.

  • Apple silently opted you into “Privacy Preserving Ad Measurement”. This an Orwellian misrepresentation, as your browser is tracking you on behalf of advertisers, just as Google Chrome is doing with its Topics API. Firefox is equally guilty of this (PDF) and unrepentant. Even Google, the most voyeuristic of the surveillance-industrial complex, asked for permission before enabling this in Chrome, albeit with wildly misleading wording because no one does dark patterns quite as smugly as don’t do be evil Google.

  • Apple silently opted you in to “Enhanced Visual Search”, where it uploads fingerprints of landmarks in your photos to its server. It claims to use differential privacy and homomorphic encryption to make this privacy compliant, but this still leaks information, even if Apple’s implementation were perfectly bug-free (given the abysmal track record of Apple QA of late, this would require heroic levels of credulity).

  • They did it also for “Improve Search"—Seeing a pattern here yet?

Here are the settings you need to review and change from their privacy-invading defaults, in chronological order of when they were introduced. Apple also has the nasty habit of silently turning them back on, so you will need to check this list regularly. You will also need to set these on each device separately.

iOS and iPadOS

  • Disable the IDFA altogether and do not allow apps to ask for it:
    • Settings / Privacy & Security / Tracking / Allow Apps to Request to Track / (turn off)
  • Disable Apple’s own Ad network tracking:
    • Settings / Privacy & Security / Apple Advertising / Personalized Ads / (turn off)
  • Disable Sharing of information with Apple, including Siri recordings:
    • Settings / Analytics & Improcements / (disable all of them)
  • Private Click Measurement:
    • Settings / Apps / Safari / Advanced / Privacy Preserving Ad Measurement / (turn off)
  • Improve Search:
    • Settings / Search / Help Apple Improve Search
    • Settings / Apps / Safari / Search / Search Engine Suggestions / (turn off)
    • Settings / Apps / Safari / Search / Safari Suggestions / (turn off)
  • Visual Search:
    • Settings / Apps / Photos / Enhanced Visual Search / (turn off)
  • Journal App:
    • Settings / Privacy & Security / Journaling Suggestions / (turn them all off)

macOS

  • Disable analytics:
    • System Settings / Privacy & Security / Analytics & Improvements / (turn them all off)
    • Sign in to account.apple.com, then Privacy / iCloud Analytics / Share iCloud analytics / (turn off)
    • This might also be a good time to request export of all the data Apple holds on you
  • Disable Apple’s Ad tracking:
    • System Settings / Privacy & Security / Apple Advertising / Personalized Ads / (turn off)
  • Disable Siri:
    • System Settings / Apple Intelligence & Siri / Siri / (turn off)
    • System Settings / Apple Intelligence & Siri / Siri history / Delete Siri & Dictation History / (click on the button)
  • Private Click Measurement:
    • Safari / Settings / Advanced / Allow privacy-preserving measurement of ad effectiveness / (turn off)
  • Improve Search:
    • System Settings / Accessibility / Motor / Voice Control / Improve assistive voice features / (turn off)
    • System Settings / Spotlight / Siri Suggestions / (turn off)
    • System Settings / Spotlight / Help Apple Improve Search / (turn off)

Further actions

Ideally, change your default browser to something better, like Vivaldi or LibreWolf.

Stop iMessage from using insecure unencrypted SMS as a fallback (warning: this setting is buggy and often ignored):

  • on iOS: Setting / Apps / Messages / Send as Text Message / (turn off)

Better yet, ditch both SMS and iMessage for Signal or WhatsApp, who do not have an unencrypted option to snare you. See also this Signal hardening checklist.

Install Little Snitch, an outbound firewall you can use to control what sites apps can connect to.

Disable Apple Intelligence.

Ultimately, switch to Linux and GrapheneOS or LineageOS.

Getting the Hasivo F1100W-4SX-4XGT switch to work

TL:DR The missing (documentation) for this otherwise fine switch

I was first exposed to Cisco network equipment in 1995, and for a long time I used their gear for my home network. Then around 2012 I switched to Juniper switches for their more sane management interface and more reasonable prices. Neither have WiFI equipment that’s really appropriate for a home setting, however, with onerous licensing terms or crackpot schizophrenic hardware like my old Cisco 877W that was one half ADSL router and one half WiFi AP (coexisting uneasily in the same physical box with separate management interfaces).

By the time I got fed up with single consumer APs, their lack of coverage and tendency to burn out within a year due to inadequate power supplies, I bit the bullet, went with the Ubiquiti Networks UniFi solution that I use to the day. At one point I considered switching to TP-Link Omada, but procrastination paid off, and I dodged a bullet.

Unfortunately, Ubiquiti doesn’t have switches with both SFP+ interfaces (for fiber optic connections) and 10G-BaseT (for copper like on my Mac Studio), other than the expensive, bulky and non-fanless Pro HD 24. While you can easily get 10G-BaseT copper SFP+ modules, the power draw of a 10G-BaseT port is actually more than the nominal power capacity of a SFP+ port and in my experience they are unreliable. For a while, I used the ZyXEL XGS1250-12, which has 3 2.5G/10G copper and one SFP+, but I would prefer a switch with more SFP+ ports.

After digging through reviews, and ServeTheHome, I found out about Hasivo switches, an inexpensive Chinese brand offering great value for money and interesting port configurations. Their F1100W-4SX-4XGT offers 4x SFP+ and 4x 2.5G/10G copper ports for £152.39 plus VAT, so I ordered one. When I received it, I plugged it into my home office Ubiquiti USW Pro Max 16 PoE with a Ubiquiti 10G DAC cable, plugged my Mac, and everything just worked as it should.

The Power and RET LEDs were flashing red and green, however. Furthermore, this is supposed to be a L3-capable switch with a Web UI, but no DHCP request or IP appeared in my UniFi console or IP address (spoiler: it’s 192.168.0.1, and DHCP is not enabled by default). The switch did not include any documentation, there is nothing available on the Hasivo site, not even in Chinese (they have documentation links, but they point to a completely different product, and even then are largely useless).

Here’s how I got it to work, using information gleaned from various Internet forums:

  1. First, get a Cisco-style RJ45 serial console cable like the Cable Matters one, hook it to the Console port on the Hasivo.
  2. Start a terminal session, in my case on Linux:
chown uucp:uucp /dev/ttyUSB0
cu -l /dev/ttyUSB0 -s 38400
  1. The login is admin and the password is admin

  2. The terminal console UI is a knockoff of the Cisco IOS CLI:

    1. enable to enter administrator mode
    2. conf t to enter configuration mode
    3. interface vlan1 to configure the admin interface
    4. ip address 10.254.254.115/8 (or whatever you want it to be)
    5. exit to go back to the interface level
    6. exit to go back to the global config level
    7. show interface brief to verify the config was applied correctly
    8. copy running-config startup-config to make the changes persistent
  3. You can now point your web browser to http://10.254.254.115/

Some braver people than myself are attempting to get OpenWRT running on the switch, but they doesn’t seem to have succeeded yet.

Fiber for your home network

TL:DR Fiber as the backbone of your home network is easier than you’d think

My apartment, like many, is elongated. The living room is on one end, the bedrooms (one of which is my home office), on the other side. This makes it hard to cover both sides with a single WiFi access point, or to have uniform Internet access speed on the wired network. I have a semi-pro Ubiquiti UniFi network of WiFi access points and switches, which makes it realtively easy, but only if you have good backhaul connectivity between the APs.

For the longest time I used G.hn powerline networking bridges made by Devolo. Unfortunately, powerline is at least as unreliable as wireless networking and this made for frequent brown-outs requiring unplugging the Devolo Magic 2 boxes to power-cycle them. I know Devolo doesn’t make the actual PLC controllers and their firmware, probably made by Broadcom from the MAC addresses, but surely they could implement something as simple as a watchdog timer to reboot the PLC if no heartbeats are seen in a while?

At any rate, at some point I flipped the bozo switch on the Devolos and decided it was long past time to install proper Ethernet across the two halves of the apartment, which is easier said than done in a rental. To make things more complex, my Internet connection that used to be absymal Vodafone/BT OpenReach VDSL ending in my office was upgraded to a fiber ISP but this terminated by the door in half-way no-man’s land.

This was around the time I was experimenting with 10G Ethernet in the core of my home network, using Ubiquiti’s relatively inexpensive (for the time) USW-Aggregation switches with 8 SFP+ 10G ports. Speaking of which, while you can buy 10GBase-T SFP+ modules allowing you to use copper 10G Ethernet like the one on my Mac Studio, their power draw exceeds the specs of the SFP+ standard and they are unreliable, stick with fiber or use a switch with actual 10G ports (in my case a ZyXEL XGS1250-12, although it has an unfortunate tendency to overheat).

Contrary to what you may think, multimode fiber is much thinner (thus more discreet) and far more flexible than copper Ethernet cable (fiber above in the picture, copper below).

Fiber and copper cable compared

I conceived the idea of running a 30m pre-terminated fiber cable, made by a French company, as it turns out, along the crown molding in the ceiling, held in place with transparent plastic 3M Command hooks originally meant for holding Christmas lights, and easily removed without damage to the paintwork (this is a rental, remember).

Fiber cable on the ceiling

I had to run it along a snaking route in red to stay along the crown molding, but even with my tyro DIY skills it only took a couple of hours to set up and is barely visible unless you know to look for it. While I don’t actually have any 10G devices in my living room yet, I do have a WiFi7 access point and it won’t be bottlenecked by the Ethernet network.

Floor plan

I still have a Devolo link between my office and the AP in my bedroom, but that’s a much shorter distance and much less unreliable.