30/05/2025

Notes on installing Matlab on Kubuntu 25.04

 I don't use matlab a lot, which means I keep running into the same snags on installing.

No. 1: "error while loading shared libraries and libexpat.so.1: file too short" during install. 
That's a broken links issue that can be resolved by setting the X and K flags during unzipping 
unzip -X -K matlab_R<yourversion>.zip

Source: MATLAB answers

No. 2: Matlab crashed while trying to load the addon explorer, with some complaint about memory issues.
Apparently a Java issue to do with the open files limit (see this discussion). 

I simply increased the system-wide hard limit, but would probably try doing it via ulimit -n first next time.

10/05/2025

Passive aggressive Easter egg of the week

import seaborn as sns
import numpy as np
data = np.random.randint(0,10,(50,5))
sns.violinplot(data,palette='jet')


29/03/2025

How to get the Logitech R400 Presenter to work with Okular

 

The Logitech R400 has been around for a while, it's a solid, affordable presenter (with a slightly crappy laser, though) and you can get a good deal for it online. 

To have it fully work with a PDF presentation on Okular - i.e. use all four buttons - I had to do some simple tweaks.

First, find out what the buttons do by running xev in a terminal. 

(1) and (2) are mapped on 'Prior' and 'Next' and work out of the box. (3) is supposed to toggle presentation mode and alternates between F5 and Escape. (4) is meant to toggle a black screen and transmits a period, '.'

The quickest fix is to just redefine Okular's keyboard shortcuts in the Settings menu. I set the alternate shortcut for 'Switch Blackscreen Mode' to '.' and the one for Presentation to F5. Note that this disables the default 'Reload' shortcut, but Okular usually reloads automatically, so it's not too bad.

The alternative is to remap the presenter input from F5 to eg. F4, which is not conflicting with Okular's default scheme. I googled a bit (see this handy Medium post) and found input-remapper, which is in the Ubuntu repos and comes with a polished  and quite self-explanatory GUI.



16/07/2024

An ugly method to get matplotlib to add transparency to streamplots



Well, they said in the API docs: "This container will probably change in the future to allow changes to the colormap, alpha, etc. for both lines and arrows, but these changes should be backward compatible." but it looks like there's still a lot broken in the backend (see e.g. this GitHub issue).

So here's a very ugly workaround, based off one of the official streamplot examples
We plot some kind of velocity field, with the absolute speed in a background colour map, and alpha, width and colour mapping of the streamlines set by the speed. To get the alpha, we use an intermediate greyscale colourmap, and retroactively feed those grey values into the colour map and alpha values we want to actually use.  (TL;DR: I don't understand all the details, but it works)

from matplotlib import pyplot as plt
from matplotlib import cm
import numpy as np

fig,ax=plt.subplots(1)

w = 3
Y, X = np.mgrid[-w:w:100j, -w:w:100j]
U = -1 - X**2 + Y
V = 1 + X - Y**2
speed = np.sqrt(U**2 + V**2)
alph = (speed-speed.min())/(speed.max()-speed.min())
lw=4*alph
speed = np.sqrt(U**2 + V**2)
im=ax.imshow(speed,extent=[-w,w,-w,w],cmap=cm.bone_r)
stream=ax.streamplot(X, Y, U, V, density=0.6,linewidth=lw,color=alph,cmap=cm.gray)

#populate the colours list via draw()
fig.canvas.draw()

cols=stream.lines.get_colors()
alphas=cols[:,0] #copy alphas
cols=cm.afmhot(cols[:,0])#apply colour map
cols[:,3]=alphas #apply alphas
stream.lines.set_colors(cols)

#since the 'arrows' collection of the streamplot doesn't work,
#we access the arrow props via the axes' patch list.
for p in ax.patches:
    c=p.get_ec()#copy edge colour
    col=list(cm.afmhot(c[0]))[:3]+[c[0]]
    p.set_ec(col)
    p.set_fc(col)

plt.colorbar(im,label='speed')
plt.savefig('myfigure.png')
plt.show()

16/02/2023

Turning a LaTeX beamer presentation with movies into a portable PPTX via LibreOffice.

Use case: your talk venue insists you use their computer. And PowerPoint.

You'll need pdftoppm (or similar, comes with poppler) and FFMPEG if your movies still need conversion. And a recent Impress (mine was LO 7.4.4.2). Older versions crashed on saving as PPTX if there were embedded media. 

First, create page images, in bash:
pdftoppm mypresentation.pdf img -png -r 250

The output is a series of img-1.png, ...,img-n.png. Resolution is here 250.

If necessary, convert all movies (let's assume they're .avi) to something PowerPoint/Impress will digest. Other containers and codecs might work as well, but my Windows failsafe is WMV2.

ffmpeg -i input.avi -codec:v wmv2 -b:v 2000k output.wmv 

Batch conversion:
for m in *.avi; do ffmpeg -i $m -codec:v wmv2 -b:v 2000k ${m%avi}wmv; done 

Start a new Impress presentation, and add a photo album via the Insert->Media menu (see screenshot).

At this point it's a good idea to protect position and size for all images in the slide (Properties sidebar). This is probably scriptable, but I'm not researching it today. Add the movies on top of the slide images. Note: Videos added via drag and drop from a file browser will get linked and not embedded into the PPTX. To embed, you need to go via the Insert->Audio or Video... dialog. Save as PPTX.

Note: I tried importing PDFs directly into Impress but that only opened them in Draw . 

Customized matplotlib styles where python can find them.

 Bugged me for a while. Turns out matplotlib is quite nitpicky about exact locations and file extensions. We assume the style file is called 'mystyle.mplstyle'.

import matplotlib as mpl
import matplotlib.pyplot as plt

Find the configdir of your matplotlib:

mpl.get_configdir()

Usually it's ~/.config/matplotlib. Put the style file in a subfolder named stylelib.

plt.style.use('mystyle') should work now.

How to create your own style?

The syntax is the same as in a matplotlibrc file, so find one (e.g. via mpl.matplotlib_fname()), copy and paste. 

01/02/2023

How to get a matplotlib colormap into inkscape.

Inelegant, but works.

TL;DR: let python create an SVG file with a rectangle that uses the colormap as a gradient. Open, import or copy in inkscape.

Python code (this one gives coolwarm012.svg in your working directory):

ncol=12 #number of stops
grad='coolwarm' #name of matplotlib gradient
from matplotlib import colors, cm
import numpy as np

stopstr="""  <stop
    style="stop-color:%s;stop-opacity:1"
    offset="%.4f"
    id="stop%03d" />"""

init="""<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<svg
   width="1cm"
   height="1cm"
   viewBox="0 0 1 1"
   id="svg">   
  <defs id="defs">
    <linearGradient
       id="%s%03d">
"""%(grad,ncol)

exit="""
</linearGradient>
</defs>
<rect
       style="fill:url(#%s%03d)"
       id="rectangle"
       width="1"
       height="1"
       x="0"
       y="0"/>
</svg>"""%(grad,ncol)

cmap = cm.get_cmap(grad, ncol)   
spac=np.linspace(0,1,ncol)

#saves  <gradientname+numberofstops>.svg with a 1x1cm^2 rectangle using the gradient 
with open('%s%03d.svg'%(grad,ncol),'w') as f:
    f.write(init)
    for i in range(cmap.N):
        rgba = cmap(i)
        hexcol = colors.rgb2hex(rgba)
        f.write(stopstr%(hexcol,spac[i],i))
    f.write(exit)

Some code taken from https://stackoverflow.com/questions/33596491/extract-matplotlib-colormap-in-hex-format

18/02/2022

How to convert a LaTeX beamer presentation with videos to PPTX using Python

Update: The crash issue appears to be fixed in recent LibreOffice. So there is really no reason to do all of the below any more, hooray. See updated howto.

First, why on earth would you want to do that?

The American Physical Society insists that you upload PPTX slides to their conference server and LibreOffice crashes every time I try to import a movie. The developers on the LO forums say it's not their problem because it works with ODP. Glad you asked.

You'll need pdftoppm (or similar, comes with poppler) and the python-pptx and PIL modules. Also, FFMPEG if your movies still need conversion.

First, create page images, in bash:
pdftoppm mypresentation.pdf img -png -r 250

The output is a series of img-1.png, ...,img-n.png. Resolution is here 250.

If necessary, convert all movies to something PowerPoint will digest. Other containers and codecs might work as well, but my Windows failsafe is WMV2.

ffmpeg -i input.avi -codec:v wmv2 -b:v 2000k output.wmv

Batch conversion:

for m in *.avi; do ffmpeg -i $m -codec:v wmv2 -b:v 2000k ${m%avi}wmv; done

We'll assume that we have movie1.wmv and movie2.wmv on slide 2 and movie3.wmv on slide 5.

Now for the tedious bit - determine the bounding box (left, top, width, height, in px) for each movie on the img-1.png and img-5.png slide images. I just used gwenview's crop tool.

Now for the python script:

from pptx import Presentation
from pptx.util import Inches
from glob import glob
from PIL import Image

prs = Presentation()
prs.slide_width = Inches(16) #assuming 16:9 aspect ratio
prs.slide_height = Inches(9)

blank = prs.slide_layouts[6]

imgs = sorted(glob("img*.png"))
w,h=Image.open(imgs[0]).size
# conversion factor for pixel values to inches.
pxtoin=16./w
slides={}

# did the movie-slide attribution via some kind of messy dictionary/list construction. Use the top,left,width,height values determined above.
movies={4:[['movie3.wmv',[626,323,1263,708]]],
        1:[['movie1.wmv',[155,324,710,710]],['movie2.wmv',[1163,324,710,710]]]}

for i in range(len(imgs)):
   slides[i] = prs.slides.add_slide(blank)
   # add the slide contents as bitmaps
   pic = slides[i].shapes.add_picture(imgs[i], 0, 0, height=prs.slide_height)
   try:
      for it in movies[i]:
         movie = slides[i].shapes.add_movie(it[0],*[Inches(r*pxtoin) for r in it[1]])
   except KeyError: pass

prs.save("test.pptx")



This got me a minimum working presentation when I tested it on a remote Windows machine, apart from the fact that the movies had ugly speaker icons instead of poster images. There's a poster_frame_image keyword in add_movie where you can provide an image filename, but it didn't seem to change anything.

Sources:

The python-pptx documentation has a lot of basic use cases. https://python-pptx.readthedocs.io/










23/01/2020

Installing external python modules for Blender

Blender comes with its own Python, and it's not necessarily your system's, so just symlinking system packages like matplotlib is risky and cumbersome.
Some Googling Stackoverflow provided me with the following approach:
  • create a (temporary) virtual environment (VE) linking against Blender's python executable
  • within that VE, pip install the packages you need, but into a folder in Blender's Python path (e.g. the one in ~/.config)
  • I'd keep the VE, but Blender doesn't need it to be active to run.

In code (I keep Blender in ~/.local/bin, adapt to your setup), for matplotlib (You might have to run blender once at first to create the file structure in .config):
virtualenv --python=~/.local/bin/blender-2.81a/2.81/python/bin/python3.7m ve-blender
source ve-blender/bin/activate
mkdir -p ~/.config/blender/2.81/scripts/modules/
pip install --upgrade -t ~/.config/blender/2.81/scripts/modules/ matplotlib
deactivate


Run Blender.

Update: When I tried to do this with pickle5,  gcc failed via "Python.h: No such file or directory".

Fix: within the active VE, determine the Blender Python version (python -V).
Download the respective sources (e.g. 3.7.7) from www.python.org. and unpack.
Copy the contents of the Include directory to ~/.local/bin/blender-2.81/2.81/python/include/python3.7m/ (adapt versions and paths).

Next update:

Broken pip after initializing the virtual environment. Fixable via pip's bootstrap installer:
curl -sS https://bootstrap.pypa.io/get-pip.py | python3

Sources:
https://stackoverflow.com/questions/1534210/use-different-python-version-with-virtualenv
https://blender.stackexchange.com/questions/132278/how-to-make-use-of-custom-external-python-modules-in-blender-or-an-add-on-on-lin

https://blender.stackexchange.com/questions/81740/python-h-missing-in-blender-python 

https://stackoverflow.com/questions/49478573/pip3-install-not-working-no-module-named-pip-vendor-pkg-resources

26/10/2019

IcyBox IB-RD3621U3 RAID1 with Ubuntu 16.04

RAID is not backup, yeah, yeah, yeah…
Still, my laptop is bursting its 512GB SSD and anything is better than my current backup method of randomly distributing copies of important data on external USB drives in various states of health.

I settled on a mid price solution, two 4TB Seagate Barracuda and an IcyBox  IB-RD3621U3 RAID enclosure, for about 250€ in total.  Rsynced to a 4TB WD mobile drive I keep in daily use.
Setup procedure:
  1. S.M.A.R.T. self test. Note: the IcyBox board is not Smart capable. To do a full initial self test on the drives, plug them into a desktop SATA. (smartctl -d sat provides output, but the test will fail)
  2. Disk installation in the drive: note - there are handy stickers 'HDD1' and 'HDD2' in the box. Label your disks to correspond with the HDD1 and 2 labels on the front of the IcyBox.
  3. RAID mode:
    Switch the box off (1). Set the pins under MODE (2) to RAID1. Hold OK (3) pressed, switch the box on (1) and keep holding OK until the LED band at the front has stopped blinking in red (should take about 10 seconds)
     
  4. Connect to a computer. The box should now register as a single drive in GParted, which you can format at will.
  5. Note: GParted via the Dell XPS USB-C dongle hell didn't work too well, failed at creating the partition table.


11/06/2019

KCM Wacom settings on (K)ubuntu 16.04

There's a handy KDE System Settings module for configuring Wacom tablets - calibration, screen areas, you name it. I was rather keen to have it handle my old Graphire4 on a multi screen setup on my work desktop (Ubuntu 16.05).
Sadly, the wacomtablet package (https://github.com/KDE/wacomtablet) seems to have been dropped from the repos between 14.04 and 19.04 due to ongoing bugfixes in dependencies. They do offer instructions for manual compiling - with dire warnings that there'd there be no easy uninstall. 'make uninstall' worked nicely, though.
However, I kept running into missing headers etc., until I found the one version that worked, 2.9.82.

Command line log:
wget https://github.com/KDE/wacomtablet/archive/v2.9.82.zip 
unzip v2.9.82.zip 
cd wacomtablet-2.9.82 
mkdir build 
cd build  
cmake ../ -DCMAKE_INSTALL_PREFIX=/usr -DCMAKE_BUILD_TYPE=Release -DKDE_INSTALL_USE_QT_SYS_PATHS=ON 
make 
sudo make install

Et voilà!

23/04/2019

XPS13 webcam doesn't wake up (BIOS fix)

The XPS13 9370 (summer 2018's model) webcam is horrible anyway (still stuck at the bottom of the display). However, it didn't help that it stopped working when the laptop resumed after suspend (all Kubuntu versions I used from 18.04 upwards).
According to this Dell support discussion, it was a BIOS problem that was fixed in BIOS 1.5.0 - I had 1.3.3. Updating seems to have fixed the problem.
Steps:
Get the current BIOS installer from here.
Put it on a flash drive (unless you've got a USB-C one, the Dell dongle works. MicroSDs don't)
Plug in your power supply.
Press F12 during re-boot, select BIOS Flash Update. There should be a drop-down menu top left on the next screen that lets you select and browse the flash drive. Pick the EXE installer. Install. Reboot.

do-release-upgrade or don't

I couldn't resist the lure of the Dingo, but it took me some time to get there. Long story short: do-release-upgrade is a picky bitch. It kept and kept and kept coming up with a "Your python3 install is corrupted. Please fix the '/usr/bin/python3' symlink." message. That message is produced by a script downloaded into /tmp during the installation process and IMO *should* just make sure that the /usr/bin/python3 symlink points to a correct python binary (might have to be 3.6). After all, the update scripts have a "#!/usr/bin/python3" header.
However, over the years, people have had to resort to the weirdest workarounds:
  • Make sure /usr/bin/python and /usr/bin/python3 point to python3.6
  • Reinstall default python3 
  • Purge anything done by update-alternatives (sudo update-alternatives --remove-all python)
  • Purge python2.x
Reader, I tried all of them.
update-alternatives --display python
sudo update-alternatives --remove-all python
sudo apt install --reinstall ubuntu-release-upgrader-core
sudo apt install --reinstall python3
sudo ln -sf /usr/bin/python3.6 /usr/bin/python
sudo ln -sf /usr/bin/python3.6 /usr/bin/python3
sudo apt-get remove --purge python2.7-minimal
In the end, purging python2.7 did the trick. That involved uninstalling inkscape and bits of TeXlive, but it was a small list of dependencies that could be reinstalled after the update. Still, a mess. Serves me right for not bothering to do a clean reinstall.

11/04/2019

articletitle, article-title, gnah!

Joseph Wright is an awesome person who provides handy biblatex styles for numerous journals.
Some of those require article titles in the submission reference list which are later removed for publication, so it makes sense to put in a switch when you load the package:
\usepackage[style=science,articletitle=true]{biblatex}
(the switch is defined in the science.bbx file, btw.) All hunky dory,  except for the fact that the most recent texlive shipping with ubuntu (TeXLive 2018.20180824) still provides biblatex-science from 2016, where the switch is called article-title. I hope the newest versions get into TexLive soon, apparently he's done quite a bit of cleanup.
https://github.com/josephwright/biblatex-science/issues/1
Update: Fixed in the TeXlive 2018.20190227 that came with the Dingo upgrade. Gnah. Now my work desktop and laptop have different versions of the script.

25/03/2019

Help! Kile is stuck!

"The file x.kilepr cannot be opened as it does not appear to be a project file." Which didn't stop Kile from "scanning project files", which in turn blocked the entire interface. All in vain, I might add, since the project file had moved anyway, so there was nothing to scan in a folder that didn't exist any more. An existential crisis ensued: it is almost midnight, after all, and I'm in mid deadline panic.
Ah well, let's find kilerc, which happens to be in ~/.config/ (if there is a method to KDE's config folder organisation's madness I haven't found it yet). The culprit was in the FilesOpenOnStart section:

[FilesOpenOnStart]
DocsOpenOnStart0[$e]=/path/to/file1.tex
DocsOpenOnStart1[$e]=/path/to/file2.tex
EncodingsOfDocsOpenOnStart0[$e]=UTF-8
EncodingsOfDocsOpenOnStart1[$e]=UTF-8
NoDOOS=2
NoPOOS=1
ProjectsOpenOnStart0[$e]=/this/path/is/invalid.kilepr

I just had to adapt the numbers and paths of docs and projects open on start after all. 

23/08/2018

Zenbook data rescue

RIP, Medusa (2013 Zenbook). I believe her power socket is finally dead.

Which was a bit of a bummer because I'm not too regular in my backups and the old Zenbooks feature non-standard 18pin-connectors instead of the more recent M.2/NGFFs. Ah well. There are adaptors for everything.
There were exactly 2 drive types with this form factor, SanDisk's SASA5JK and ADATA's XM11 and I got a board that transfers to a standard SATA and can then be stuck in a USB drive case like the one pictured above (I got the adaptor from here, but there are other suppliers as well).

16/05/2018

KDE freezes on Special Window/Application Settings menu

I wanted to disable Global Shortcuts for Blender and happened on a nasty bug on Kubuntu 16.04 (applies to all windows).

The standard method is to go via right click on the title bar, 'More Actions' and 'Special Application (Window) Settings', clicking which completely froze my desktop. It wasn't reproducible on my laptop running 18.04, so I just note the workaround: don't ever click on those menu entries. Save your work more often. Change Window Rules via System Settings -> Window Management -> Window Rules.

26/03/2018

Spyder 3.2.8, pip3 and and PyQt5

I recently ran into segfaults on all my computers after upgrading spyder via pip3. Lots of Qt5 error messages. The trouble seems to be PyQt5-5.10.1; spyder 3.2.8 requires at least 5.10.0 (which happens to work). Thus pip3 install --upgrade PyQt5==5.10.0 spyder 
solved my issue.
Note to self: pip3 install PyQt5== is a handy way to list available versions. Sadly, it's deprecated (--use-deprecated=legacy-resolver).
Second note to self: There's also spyder==4.0.0b1 which seems to work with newer PyQts. Good thing I switched to virtual environments.

23/03/2018

Moving your thunderbird data to a custom location

Thunderbird's profile data is stored in ~/.thunderbird by default.
If that's inconvenient, i.e. due to disk quota limits, here's is how to move the profile folder. It will have some random name, we'll call it pr0f1le.default, and ~/.thunderbird contains a file profiles.ini and a folder named pr0f1le.default.
Here, I've moved the profile folder to e.g. /media/data/thunderbird/pr0f1le.default and changed profiles.ini to (changes in bold):
[General]
StartWithLastProfile=1

[Profile0]
Name=default
IsRelative=0
Path=/media/data/thunderbird/pr0f1le.default

Default=1

IsRelative has to be 0 because I've given Thunderbird an absolute path to the new location.

08/09/2017

How to get okular to play nice with pretty much all embedded movies

The trick is to have VLC play them, because vlc plays basically everything.
Install via apt-get either phonon4qt5-backend-vlc or phonon-backend-vlc or both.
In System Settings -> Audio and Video, prefer 'Phonon VLC' in the backend tab. If in doubt, restart Okular.