16/07/2024

An ugly method to get matplotlib to add transparency to streamplots



Well, they said in the API docs: "This container will probably change in the future to allow changes to the colormap, alpha, etc. for both lines and arrows, but these changes should be backward compatible." but it looks like there's still a lot broken in the backend (see e.g. this GitHub issue).

So here's a very ugly workaround, based off one of the official streamplot examples
We plot some kind of velocity field, with the absolute speed in a background colour map, and alpha, width and colour mapping of the streamlines set by the speed. To get the alpha, we use an intermediate greyscale colourmap, and retroactively feed those grey values into the colour map and alpha values we want to actually use.  (TL;DR: I don't understand all the details, but it works)

from matplotlib import pyplot as plt
from matplotlib import cm
import numpy as np

fig,ax=plt.subplots(1)

w = 3
Y, X = np.mgrid[-w:w:100j, -w:w:100j]
U = -1 - X**2 + Y
V = 1 + X - Y**2
speed = np.sqrt(U**2 + V**2)
alph = (speed-speed.min())/(speed.max()-speed.min())
lw=4*alph
speed = np.sqrt(U**2 + V**2)
im=ax.imshow(speed,extent=[-w,w,-w,w],cmap=cm.bone_r)
stream=ax.streamplot(X, Y, U, V, density=0.6,linewidth=lw,color=alph,cmap=cm.gray)

#populate the colours list via draw()
fig.canvas.draw()

cols=stream.lines.get_colors()
alphas=cols[:,0] #copy alphas
cols=cm.afmhot(cols[:,0])#apply colour map
cols[:,3]=alphas #apply alphas
stream.lines.set_colors(cols)

#since the 'arrows' collection of the streamplot doesn't work,
#we access the arrow props via the axes' patch list.
for p in ax.patches:
    c=p.get_ec()#copy edge colour
    col=list(cm.afmhot(c[0]))[:3]+[c[0]]
    p.set_ec(col)
    p.set_fc(col)

plt.colorbar(im,label='speed')
plt.savefig('myfigure.png')
plt.show()

16/02/2023

Turning a LaTeX beamer presentation with movies into a portable PPTX via LibreOffice.

Use case: your talk venue insists you use their computer. And PowerPoint.

You'll need pdftoppm (or similar, comes with poppler) and FFMPEG if your movies still need conversion. And a recent Impress (mine was LO 7.4.4.2). Older versions crashed on saving as PPTX if there were embedded media. 

First, create page images, in bash:
pdftoppm mypresentation.pdf img -png -r 250

The output is a series of img-1.png, ...,img-n.png. Resolution is here 250.

If necessary, convert all movies (let's assume they're .avi) to something PowerPoint/Impress will digest. Other containers and codecs might work as well, but my Windows failsafe is WMV2.

ffmpeg -i input.avi -codec:v wmv2 -b:v 2000k output.wmv 

Batch conversion:
for m in *.avi; do ffmpeg -i $m -codec:v wmv2 -b:v 2000k ${m%avi}wmv; done 

Start a new Impress presentation, and add a photo album via the Insert->Media menu (see screenshot).

At this point it's a good idea to protect position and size for all images in the slide (Properties sidebar). This is probably scriptable, but I'm not researching it today. Add the movies on top of the slide images. Note: Videos added via drag and drop from a file browser will get linked and not embedded into the PPTX. To embed, you need to go via the Insert->Audio or Video... dialog. Save as PPTX.

Note: I tried importing PDFs directly into Impress but that only opened them in Draw . 

Customized matplotlib styles where python can find them.

 Bugged me for a while. Turns out matplotlib is quite nitpicky about exact locations and file extensions. We assume the style file is called 'mystyle.mplstyle'.

import matplotlib as mpl
import matplotlib.pyplot as plt

Find the configdir of your matplotlib:

mpl.get_configdir()

Usually it's ~/.config/matplotlib. Put the style file in a subfolder named stylelib.

plt.style.use('mystyle') should work now.

How to create your own style?

The syntax is the same as in a matplotlibrc file, so find one (e.g. via mpl.matplotlib_fname()), copy and paste. 

01/02/2023

How to get a matplotlib colormap into inkscape.

Inelegant, but works.

TL;DR: let python create an SVG file with a rectangle that uses the colormap as a gradient. Open, import or copy in inkscape.

Python code (this one gives coolwarm012.svg in your working directory):

ncol=12 #number of stops
grad='coolwarm' #name of matplotlib gradient
from matplotlib import colors, cm
import numpy as np

stopstr="""  <stop
    style="stop-color:%s;stop-opacity:1"
    offset="%.4f"
    id="stop%03d" />"""

init="""<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<svg
   width="1cm"
   height="1cm"
   viewBox="0 0 1 1"
   id="svg">   
  <defs id="defs">
    <linearGradient
       id="%s%03d">
"""%(grad,ncol)

exit="""
</linearGradient>
</defs>
<rect
       style="fill:url(#%s%03d)"
       id="rectangle"
       width="1"
       height="1"
       x="0"
       y="0"/>
</svg>"""%(grad,ncol)

cmap = cm.get_cmap(grad, ncol)   
spac=np.linspace(0,1,ncol)

#saves  <gradientname+numberofstops>.svg with a 1x1cm^2 rectangle using the gradient 
with open('%s%03d.svg'%(grad,ncol),'w') as f:
    f.write(init)
    for i in range(cmap.N):
        rgba = cmap(i)
        hexcol = colors.rgb2hex(rgba)
        f.write(stopstr%(hexcol,spac[i],i))
    f.write(exit)

Some code taken from https://stackoverflow.com/questions/33596491/extract-matplotlib-colormap-in-hex-format

18/02/2022

How to convert a LaTeX beamer presentation with videos to PPTX using Python

Update: The crash issue appears to be fixed in recent LibreOffice. So there is really no reason to do all of the below any more, hooray. See updated howto.

First, why on earth would you want to do that?

The American Physical Society insists that you upload PPTX slides to their conference server and LibreOffice crashes every time I try to import a movie. The developers on the LO forums say it's not their problem because it works with ODP. Glad you asked.

You'll need pdftoppm (or similar, comes with poppler) and the python-pptx and PIL modules. Also, FFMPEG if your movies still need conversion.

First, create page images, in bash:
pdftoppm mypresentation.pdf img -png -r 250

The output is a series of img-1.png, ...,img-n.png. Resolution is here 250.

If necessary, convert all movies to something PowerPoint will digest. Other containers and codecs might work as well, but my Windows failsafe is WMV2.

ffmpeg -i input.avi -codec:v wmv2 -b:v 2000k output.wmv

Batch conversion:

for m in *.avi; do ffmpeg -i $m -codec:v wmv2 -b:v 2000k ${m%avi}wmv; done

We'll assume that we have movie1.wmv and movie2.wmv on slide 2 and movie3.wmv on slide 5.

Now for the tedious bit - determine the bounding box (left, top, width, height, in px) for each movie on the img-1.png and img-5.png slide images. I just used gwenview's crop tool.

Now for the python script:

from pptx import Presentation
from pptx.util import Inches
from glob import glob
from PIL import Image

prs = Presentation()
prs.slide_width = Inches(16) #assuming 16:9 aspect ratio
prs.slide_height = Inches(9)

blank = prs.slide_layouts[6]

imgs = sorted(glob("img*.png"))
w,h=Image.open(imgs[0]).size
# conversion factor for pixel values to inches.
pxtoin=16./w
slides={}

# did the movie-slide attribution via some kind of messy dictionary/list construction. Use the top,left,width,height values determined above.
movies={4:[['movie3.wmv',[626,323,1263,708]]],
        1:[['movie1.wmv',[155,324,710,710]],['movie2.wmv',[1163,324,710,710]]]}

for i in range(len(imgs)):
   slides[i] = prs.slides.add_slide(blank)
   # add the slide contents as bitmaps
   pic = slides[i].shapes.add_picture(imgs[i], 0, 0, height=prs.slide_height)
   try:
      for it in movies[i]:
         movie = slides[i].shapes.add_movie(it[0],*[Inches(r*pxtoin) for r in it[1]])
   except KeyError: pass

prs.save("test.pptx")



This got me a minimum working presentation when I tested it on a remote Windows machine, apart from the fact that the movies had ugly speaker icons instead of poster images. There's a poster_frame_image keyword in add_movie where you can provide an image filename, but it didn't seem to change anything.

Sources:

The python-pptx documentation has a lot of basic use cases. https://python-pptx.readthedocs.io/










23/01/2020

Installing external python modules for Blender

Blender comes with its own Python, and it's not necessarily your system's, so just symlinking system packages like matplotlib is risky and cumbersome.
Some Googling Stackoverflow provided me with the following approach:
  • create a (temporary) virtual environment (VE) linking against Blender's python executable
  • within that VE, pip install the packages you need, but into a folder in Blender's Python path (e.g. the one in ~/.config)
  • I'd keep the VE, but Blender doesn't need it to be active to run.

In code (I keep Blender in ~/.local/bin, adapt to your setup), for matplotlib (You might have to run blender once at first to create the file structure in .config):
virtualenv --python=~/.local/bin/blender-2.81a/2.81/python/bin/python3.7m ve-blender
source ve-blender/bin/activate
mkdir -p ~/.config/blender/2.81/scripts/modules/
pip install --upgrade -t ~/.config/blender/2.81/scripts/modules/ matplotlib
deactivate


Run Blender.

Update: When I tried to do this with pickle5,  gcc failed via "Python.h: No such file or directory".

Fix: within the active VE, determine the Blender Python version (python -V).
Download the respective sources (e.g. 3.7.7) from www.python.org. and unpack.
Copy the contents of the Include directory to ~/.local/bin/blender-2.81/2.81/python/include/python3.7m/ (adapt versions and paths).

Next update:

Broken pip after initializing the virtual environment. Fixable via pip's bootstrap installer:
curl -sS https://bootstrap.pypa.io/get-pip.py | python3

Sources:
https://stackoverflow.com/questions/1534210/use-different-python-version-with-virtualenv
https://blender.stackexchange.com/questions/132278/how-to-make-use-of-custom-external-python-modules-in-blender-or-an-add-on-on-lin

https://blender.stackexchange.com/questions/81740/python-h-missing-in-blender-python 

https://stackoverflow.com/questions/49478573/pip3-install-not-working-no-module-named-pip-vendor-pkg-resources

26/10/2019

IcyBox IB-RD3621U3 RAID1 with Ubuntu 16.04

RAID is not backup, yeah, yeah, yeah…
Still, my laptop is bursting its 512GB SSD and anything is better than my current backup method of randomly distributing copies of important data on external USB drives in various states of health.

I settled on a mid price solution, two 4TB Seagate Barracuda and an IcyBox  IB-RD3621U3 RAID enclosure, for about 250€ in total.  Rsynced to a 4TB WD mobile drive I keep in daily use.
Setup procedure:
  1. S.M.A.R.T. self test. Note: the IcyBox board is not Smart capable. To do a full initial self test on the drives, plug them into a desktop SATA. (smartctl -d sat provides output, but the test will fail)
  2. Disk installation in the drive: note - there are handy stickers 'HDD1' and 'HDD2' in the box. Label your disks to correspond with the HDD1 and 2 labels on the front of the IcyBox.
  3. RAID mode:
    Switch the box off (1). Set the pins under MODE (2) to RAID1. Hold OK (3) pressed, switch the box on (1) and keep holding OK until the LED band at the front has stopped blinking in red (should take about 10 seconds)
     
  4. Connect to a computer. The box should now register as a single drive in GParted, which you can format at will.
  5. Note: GParted via the Dell XPS USB-C dongle hell didn't work too well, failed at creating the partition table.


11/06/2019

KCM Wacom settings on (K)ubuntu 16.04

There's a handy KDE System Settings module for configuring Wacom tablets - calibration, screen areas, you name it. I was rather keen to have it handle my old Graphire4 on a multi screen setup on my work desktop (Ubuntu 16.05).
Sadly, the wacomtablet package (https://github.com/KDE/wacomtablet) seems to have been dropped from the repos between 14.04 and 19.04 due to ongoing bugfixes in dependencies. They do offer instructions for manual compiling - with dire warnings that there'd there be no easy uninstall. 'make uninstall' worked nicely, though.
However, I kept running into missing headers etc., until I found the one version that worked, 2.9.82.

Command line log:
wget https://github.com/KDE/wacomtablet/archive/v2.9.82.zip 
unzip v2.9.82.zip 
cd wacomtablet-2.9.82 
mkdir build 
cd build  
cmake ../ -DCMAKE_INSTALL_PREFIX=/usr -DCMAKE_BUILD_TYPE=Release -DKDE_INSTALL_USE_QT_SYS_PATHS=ON 
make 
sudo make install

Et voilĂ !