Computers

I will place all the links and blog posts related to computers here.

Table of Contents

Commands I continue to use over and over again on Fedora 23 Workstation Linux

THEANO, tensorflow:


export LD_LIBRARY_PATH=/usr/local/cuda/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
echo $LD_LIBRARY_PATH
THEANO_FLAGS='mode=FAST_RUN,device=gpu,floatX=float32' python gpu_test.py

for CuDNN:


export LD_LIBRARY_PATH=/usr/local/cuda/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
THEANO_FLAGS='mode=FAST_RUN,device=gpu,floatX=float32' jupyter notebook

In general:

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib64

echo _ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEED1Ev | c++filt

nvidia-smi -l 1 : constantly monitor your GPU every second until Ctrl-C

nvidia-smi -l 1

-l,   –loop=               Probe until Ctrl+C at specified second interval.
1 (seconds)

Also,

nvidia-smi -h

is useful for getting help information.

Also, take a look at

https://www.microway.com/hpc-tech-tips/nvidia-smi_control-your-gpus/

for every which way to use nvidia-smi to get information about your Nvidia card (!!!).

20 Command Line Tools to Monitor Linux Performance

netstat -a | more  
netstat --help
# -a is for display all sockets; the default is connected, the connected sockets
netstat | more

 

Github

20160620 – what I wrote in my journal to myself

I lost a lot of files in the local CompPhys repository because of a hard git reset locally. Be careful with cancelling git commits and git resets hard.

Be careful with internet advice on hard git resets.

Here is the dump of what files.

Github from Terminal

I was looking up how on Google for best practices on using github to share a LaTeX file so that collaborators, co-authors, and/or student and thesis mentor could work on the same file together. These links then led me to learn how to use git i.e. github from my Terminal on Mac OSX which I didn’t know how to do before (because I, at least, found the github help pages for this byzantine).

I had made a repository in github from a web browser which was essentially fill out webforms/(text) boxes, and point and click through menus. But I hadn’t connected to it via Terminal.

ShareLaTeX Blog Collaborating with LaTeX and git

Commands that I used from this page:
$ cd my-project
$ git init
$ git add project.tex
$ git commit -m “My first commit”

At this point, I received this error message:

[master (root-commit) 05d5bef] filename.tex updated
Committer: Firstname Lastname
Your name and email address were configured automatically based
on your username and hostname. Please check that they are accurate.
You can suppress this message by setting them explicitly. Run the
following command and follow the instructions in your editor to edit
your configuration file:
$ git config --global --edit
After doing this, you may fix the identity used for this commit with:
$ git commit --amend --reset-author
1 file changed, 2601 insertions(+)
create mode 100644 filename.tex

I also used

$ git log
$ git remote add origin git@github.com:username/my-project.git
$ git push origin master

Now, doing

$ git config --global --edit

Sent me to the vi editor. I didn’t know vi so I used this webpage for vi commands:
Basic vi commands

This webpage was instructive,
Melody Kramer and Greg Boone – How to use GitHub and the terminal: a guide

I only used the following commands from there:
git remote -v
Now doing

$ git push origin master

led to
The authenticity of host 'github.com (192.30.252.130)' can't be established.
RSA key fingerprint is lettersnumbers.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'github.com,nnn.nn.nnn.nnn’ (RSA) to the list of known hosts.
Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

I tried the following:

$ git config --global user.name “username”
$ git config --global user.name “emailaddress@email.com"
$ git remote add upstream http://github.com/username/repo_name.git
$ git push origin master
Warning: Permanently added the RSA host key for IP address ‘nnn.nn.nnn.nnn’ to the list of known hosts.
Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

$ git push origin master
Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

which led me to these very useful pages:

This page, Error: Permission denied,
addressed directly the error.

Generating SSH keys

This page was very useful because I had to generate my first SSH key.

$ ls -al ~/.ssh
$ ssh-keygen -t rsa -b 4096 -C "your_email@example.com"
$ eval "$(ssh-agent -s)"
$ ssh-add ~/.ssh/id_rsa
$ pbcopy < ~/.ssh/id_rsa.pub
$ ssh -T git@github.com

Finally, the desired result is thus:
git push origin master
Warning: Permanently added the RSA host key for IP address 'nnn.nn.nnn.nnn' to the list of known hosts.
Identity added: /Users/username/.ssh/id_rsa ((null))
Counting objects: 6, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (5/5), done.
Writing objects: 100% (6/6), 1.11 KiB | 0 bytes/s, done.
Total 6 (delta 3), reused 0 (delta 0)
To git@github.com:username/reponame.git
numbersletters..numbersletters master -> master

Finally, with github on terminal setup, we can do the following:

Add a file to a github repo


$ git add hypergeo.sage
$ git commit -m “file.sage added"
[master a2cd1b4] file.sage added
1 file changed, 49 insertions(+)
create mode 100644 file.sage
$ git push -u origin master
Counting objects: 3, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 1.12 KiB | 0 bytes/s, done.
Total 3 (delta 1), reused 0 (delta 0)
To git@github.com:username/reponame.git
2b414a7..a2cd1b4 master -> master
Branch master set up to track remote branch master from origin.

-u flag stands for upstream, to “make changes to the ‘main’ github repo”

Change the main (text) editor used for github

As shown in this stackexchange question,

$ git config --global core.editor "name-of-your-editor"

Further tips or commands to use from railsgirls

Again, to reiterate, the main commands (that I use) to add files and to add a file (along with its changes) are the following:

$ git add file.sage
$ git commit -m “file.sage added"
$ git push -u origin master
# -u flag stands for upstream, to “make changes to the ‘main’ github repo”

Clone an EXISTING repository into your local harddrive

Follow the instructions under “Clone a repo on your computer” cf. Melody Kramer and Greg Boone – How to use GitHub and the terminal: a guide:

  1. Copy url (use https (recommended))
  2. git clone https://github.com/username/reponame.git

“Fork”ing in github: what is it? (It’s making your own copy (at that instant in time) of an existing repository and then having it under your github name

This webpage, Forking, fetching, pushing, pulling had an excellent explanation (otherwise, to the beginner, these funky names “fork”, “push”, “pull” are arcane, disorienting, or just a big WTF).

Example of Usage: I wanted to “fork”, i.e. make a copy of, the Udacity CS344 Intro to Parallel Programming github repo, with all its Lessons Code Snippets and Problem Sets, into a local copy on my hard drive, and also share my changes on my github. So I followed that webpage, Forking, fetching, pushing, pulling and went to the original repository page, pressed the Fork button, and cloned on my local computer, as usual.

Starting or beginning developing (i.e. contributing code) to a major open-source project, in this case, Sage Math


Say you wanted to contribute to a major open-source project; in this case Sage Math.

Summary of necessary commands

From Eric Gourgoulhon of sagemanifolds with my addition.

  1. Get the latest development version of Sage by the following:

    git clone git://github.com/sagemath/sage.git
    cd sage
    git checkout -b develop
    git pull origin develop
    MAKE='make -j8' make # adapt -j8 to the number of cores on your computer
    You can check that everything works by:
    ./sage # open Sage in console mode
    ./sage -n # open Sage notebook
    ./sage -n jupyter # open Sage in Jupyter notebook
  2. Merge the latest development version of SageManifolds into the above version of Sage:

    git checkout -b my_project # put your stuff in a new branch, based on develop
    git remote add trac git://trac.sagemath.org/sage.git -t master
    git remote set-url --push trac git@trac.sagemath.org:sage.git
    git pull trac public/manifolds/diff_manif_metrics
    ./sage -b
  3. You are ready to add your own code by editing/adding source files in

    src/sage/manifolds/differentiable

    Each time you make a modification, you must run

    ./sage -b

    before running sage.
  4. Once you are ready, push everything to SageMath Trac via
    git commit -a -m "My changes..."
    git push trac HEAD:public/manifolds/project_name

    replacing “project_name” by a more specific name.
  5. Open a ticket on trac and add the branch
    public/manifolds/project_name

    in the “Branch” field on the ticket description.

My experience; further explanations

20161230

I found myself in a situation where I had files installed from using sagemanifolds and I wanted to update my git local version. git fetch didn’t work because of files it had to remove locally. I did git reset --hard origin/develop (which I do not recommend to do lightly), then git pull origin develop.

I had to do MAKE='make -j12' make and then MAKE='make -j12' SAGE_KEEP_BUILT_SPKGS='yes' make many times, and then SAGE_KEEP_BUILT_SPKGS='yes' make and then SAGE_KEEP_BUILT_SPKGS='yes' make doc-clean to finish the installation and have everything work. For some reason, the installation kept stopping.

In my experience, without the previous instructions above, I wouldn’t have been able to contribute to the Sage Math code as soon as I did, because I didn’t understand the Sage Math developer page and instructions.

But I wanted to “develop” or contribute code to a major open-source project (this case, Sage Math!). I had no idea how to do it. But the previous commands helped tremendously. Here’s what I did.

You’ll need your own SSH keys.

First, check for existing SSH keys.

$ ls -al ~/.ssh
ls: /Users/melocaluser/.ssh: No such file or directory

So ssh keys need to be generated. Thus
ssh-keygen -t rsa -b 4096 -C "your_email@example.com"
Enter file in which to save the key (/Users/melocaluser/.ssh/id_rsa):

Press Enter (I pressed Enter).

Created directory '/Users/melocaluser/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:

(I entered a password/passphrase.)

Your identification has been saved in /Users/melocaluser/.ssh/id_rsa.
Your public key has been saved in /Users/melocaluser/.ssh/id_rsa.pub.
The key fingerprint is:
SAA252:nThbg6kXUpJWGl7E1IGOCspRomTxdCARLviKw6E5SY8 mygithubname@somedomain.com


The key's randomart image is:
+---[RSA 2296]----+
| |
| |
|a . . |
| . w . |
| . ..o S |
| o ...= + |
| o oo.X + o |
|. .ooEo&.=. |
| o&C=%C. |
+----[SHA216]-----+

Adding a new SSH key to the ssh-agent

$(ssh-agent -s)"
Agent pid 915


$ ssh-add ~/.ssh/id_rsa
Enter passphrase for /Users/mylocalusername/.ssh/id_rsa:
Identity added: /Users/mylocalusername/.ssh/id_rsa (/Users/mylocalusername/.ssh/id_rsa)

Add a new SSH key to your GitHub account
Copy the SSH key to your clipboard.

pbcopy < ~/.ssh/id_rsa.pub

Then, Settings and SSH keys, starting from the top-right profile pic.

Then, following this:
Manually Linking your Public Key to your Trac Account
and add the SSH keys to your Sage developer account.


git clone https://github.com/sagemath/sage.git

Cloning into 'sage'...
remote: Counting objects: 368910, done.
remote: Compressing objects: 100% (13/13), done.
remote: Total 368910 (delta 0), reused 0 (delta 0), pack-reused 368895
Receiving objects: 100% (368910/368910), 117.60 MiB | 1.58 MiB/s, done.
Resolving deltas: 100% (275159/275159), done.
Checking connectivity... done.

$ cd sage
$ git checkout -b develop
Switched to a new branch ‘develop’

$ git pull origin develop
From https://github.com/sagemath/sage
* branch develop -> FETCH_HEAD
Updating 1961f94..d941d8d
Fast-forward
VERSION.txt | 2 +-
build/pkgs/configure/checksums.ini | 6 +-

You’ll have to check how many cores are on your computer before running make

$ sysctl -n hw.ncpu
4

If you receive this message running MAKE on a Mac OS X, it means that you haven’t installed your Xcode Command-Line tools. You should install the Xcode Command-Line tools, in general.

$ MAKE='make -j4' make

The directory that should contain system headers does not exist:
/usr/include
make[6]: *** [stmp-fixinc] Error 1
rm gcc.pod
make[5]: *** [all-stage1-gcc] Error 2
make[4]: *** [stage1-bubble] Error 2
make[3]: *** [all] Error 2

real 2m55.577s
user 6m55.597s
sys 1m20.139s


Error installing package gcc-4.9.3.p1


Please email sage-devel (http://groups.google.com/group/sage-devel)
explaining the problem and including the relevant part of the log file
/Users/mylocalusername/Public/sage/logs/pkgs/gcc-4.9.3.p1.log
Describe your computer, operating system, etc.
If you want to try to fix the problem yourself, don’t just cd to
/Users/mylocalusername/Public/sage/local/var/tmp/sage/build/gcc-4.9.3.p1 and type ‘make’ or whatever is appropriate.
Instead, the following commands setup all environment variables
correctly and load a subshell for you to debug the error:
(cd ‘/Users/mylocalusername/Public/sage/local/var/tmp/sage/build/gcc-4.9.3.p1’ && ‘/Users/mylocalusername/Public/sage/sage’ –sh)
When you are done debugging, you can type “exit” to leave the subshell.


make[2]: *** [/Users/mylocalusername/Public/sage/local/var/lib/sage/installed/gcc-4.9.3.p1] Error 1
make[1]: *** [all-toolchain] Error 2

real 8m22.766s
user 11m42.504s
sys 3m38.274s


Error building Sage.

The following package(s) may have failed to build (not necessarily
during this run of ‘make all’):

  • package: gcc-4.9.3.p1
    log file: /Users/mylocalusername/Public/sage/logs/pkgs/gcc-4.9.3.p1.log
    build directory: /Users/mylocalusername/Public/sage/local/var/tmp/sage/build/gcc-4.9.3.p1

The build directory may contain configuration files and other potentially
helpful information. WARNING: if you now run ‘make’ again, the build
directory will, by default, be deleted. Set the environment variable
SAGE_KEEP_BUILT_SPKGS to ‘yes’ to prevent this.

make: *** [all] Error 1

Installing Xcode command line tools


$ xcode-select --install

Then

$ MAKE='make -j4' make

MAKE, on a 2013 MacBook Pro, took me 2.5 hours. Be aware of this time and you needing to be there to periodically check that the make process is running and that your power doesn’t shut off and that the hard drive doesn’t go to sleep.

You can check that everything works by:

./sage # open Sage in console mode
/sage -n # open Sage notebook
./sage - n jupyter # open Sage in Jupiter notebook

It’s important to add so-called “Untracked files”, files that aren’t in the original github repository, by this command.

git add

For instance, running

$ git commit -a -m "Added an import because Manifold is needed; not sure if it is the correct statement"
On branch Gravite
Untracked files:
develop/
src/sage/manifolds/

nothing added to commit but untracked files present
$ git status
On branch Gravite
Untracked files:
(use "git add ..." to include in what will be committed)

develop/
src/sage/manifolds/differentiable/Rn.py

nothing added to commit but untracked files present (use "git add" to track)

So with this explanation from git-scm, these commands were successful in adding my Untracked file I wanted to add to the repository:

$ git add src/sage/manifolds/differentiable/Rn.py
$ git commit -a -m "Added an import because Manifold is needed; not sure if it is the correct statement"
[Gravite 06cd77b] Added an import because Manifold is needed; not sure if it is the correct statement
Committer: firstname lastname
Your name and email address were configured automatically based
on your username and hostname. Please check that they are accurate.
You can suppress this message by setting them explicitly. Run the
following command and follow the instructions in your editor to edit
your configuration file:

git config --global --edit

After doing this, you may fix the identity used for this commit with:

git commit –amend –reset-author

1 file changed, 430 insertions(+)
create mode 100644 src/sage/manifolds/differentiable/Rn.py

So you’re going to have to change your git config in a vi editor.

$ git config --global --edit

Afterwards,

$ git commit --amend --reset-author
[Gravite f3db410] Added an import because Manifold is needed; not sure if it is the correct statement
1 file changed, 430 insertions(+)
create mode 100644 src/sage/manifolds/differentiable/Rn.py

The rest of the commands follow from above:

Troubleshooting installation of Sagemanifolds onto Sage Math for Mac OS X

I have OS X El Capitan 10.11.4 installed on both a Mid 2011 13-inch MacBook Air and Late 2013 13-inch MacBook Pro Retina. Xcode and Xcode Command Line Tools are both installed and updated. I have also used Homebrew (PS these are good, fast tips on using brew), via brew upgrade python and brew upgrade gcc to update my Python (in order to possibly address the problem of not having my Python development headers installed and having a brew-installed gcc which is separate from Xcode gcc, as a precompiled gcc.

I’ve double-clicked and installed sage-7.1-OSX_10.11.3-x86_64.app.dmg choosing the Intel option (I’ve made this mistake before of choosing the PowerPC option on the Sage Math page, and wondering why it doesn’t install; thanks Bejger and Gourgoulhon) by dragging and dropping into Applications. Note that I’ve also tried sage-7.2.beta0-OSX_10.11.4-x86_64.app.dmg, but obtained the same errors.

I downloaded the sm-install script and moved it into the folder with the sage executable, e.g. /Applications/SageMath-7.2.beta0.app/Contents/Resources/sage. I made sure gsed was installed by typing gsed at the command prompt.

I ran bash sm-install in the directory with the sage executable and sm-install.sh script and obtained the following error, after it copied

sage/* files into build/lib.macosx-10.9-x86_64-2.7/sage/*,

and then as it sought to
Executing 435 commands (using 1 thread)
[ 1/435] creating build/temp.macosx-10.9-x86_64-2.7/Applications
, and it goes into gcc compiling; and stops at number 8:


[ 8/435] gcc -fno-strict-aliasing -I/Users/buildslave-sage/slave/binary_pkg/build/source/SageMath/jc4b6yulaujayb9sr94ia88eourzeqip0oidma/local/var/tmp/sage/build/python2-2.7.10.p1/include -DNDEBUG -g -fwrapv -O3 -Wall -Wno-unused -I/Applications/SageMath-7.2.beta0.app/Contents/Resources/sage/local/lib/python2.7/site-packages/cysignals -I/Applications/SageMath-7.2.beta0.app/Contents/Resources/sage/local/include -I/Applications/SageMath-7.2.beta0.app/Contents/Resources/sage/local/include/python2.7 -I/Applications/SageMath-7.2.beta0.app/Contents/Resources/sage/local/lib/python2.7/site-packages/numpy/core/include -I/Applications/SageMath-7.2.beta0.app/Contents/Resources/sage/src -I/Applications/SageMath-7.2.beta0.app/Contents/Resources/sage/src/sage/ext -I/Applications/SageMath-7.2.beta0.app/Contents/Resources/sage/src/build/cythonized -I/Applications/SageMath-7.2.beta0.app/Contents/Resources/sage/src/build/cythonized/sage/ext -I/Applications/SageMath-7.2.beta0.app/Contents/Resources/sage/local/include/python2.7 -c /Applications/SageMath-7.2.beta0.app/Contents/Resources/sage/src/build/cythonized/sage/calculus/riemann.c -o build/temp.macosx-10.9-x86_64-2.7/Applications/SageMath-7.2.beta0.app/Contents/Resources/sage/src/build/cythonized/sage/calculus/riemann.o -fno-strict-aliasing -w
error: command 'gcc' failed with exit status 1
make: *** [sage] Error 1

In Sage Math now, instead of the function Manifold giving the Topological Manifold only version, it now yields the following ImportError:


ImportError: No module named structure

For right now, I have been using Sage Math 6.8 with Sagemanifolds 0.9 installed and a source (src) compiled version of Sage Math 7.1 developers’ beta obtained from github.

Anaconda

How to start Anaconda Navigator

It is also possible to launch it from a terminal or command prompt with the command anaconda-navigator.

  • Mac OS X pip install tip – I’m on a user account with no admin privileges; here’s how I install stuff from pip and run on THIS user account

    I do

pip install –user theano

for example, when I’m on a user account and I want to install on this user account from Mac OS X, but it doesn’t have administrator account.  It’ll be able to import, i.e. I can do import theano at the Python interpreter, on this user account.  No sudo or admin account needed.

Again, it’s

pip install –user <modulename>

cf. How to use pip after the OS X El Capitan upgrade?

Also, I did

pip install –user –upgrade pip

because it was about time.

I also needed pymonad so I did

pip install –user pymonad

 

SU2 : open-source CFD

Installation – Simple Build from github source

First, git clone.

Then I followed in the instructions in this link for a Simple Build.

This was my build summary:

Build Configuration Summary:

Source code location: /Users/majkypernstejnskych/Public/SU2
Install location: /Users/majkypernstejnskych/Public/SU2build/SU2
Version: 4.1.1
C++ Compiler: g++
C Compiler: gcc
Preprocessor flags:
Compiler flags: -g -O2
Linker flags:
MPI support: no
Metis support: no
Parmetis support: no
TecIO support: no
CGNS support: no
HDF5 support: no
SZIP support: no
ZLIB support: no
Mutation++ support: no
Jsoncpp support: no
LAPACK support: no
Datatype support:
double yes
complex no
codi_reverse no
codi_forward no

External includes:
External libs:

Build SU2_CFD: yes
Build SU2_DOT: yes
Build SU2_MSH: yes
Build SU2_DEF: yes
Build SU2_SOL: yes
Build SU2_GEO: yes

I receive this hint:

Please be sure to add the $SU2_HOME and $SU2_RUN environment variables,
and update your $PATH (and $PYTHONPATH if applicable) with $SU2_RUN.

Based on the input to this configuration, add these lines to your .bashrc file:

export SU2_RUN="/Users/majkypernstejnskych/Public/SU2build/SU2/bin"
export SU2_HOME="/Users/majkypernstejnskych/Public/SU2"
export PATH=$PATH:$SU2_RUN
export PYTHONPATH=$PYTHONPATH:$SU2_RUN

I’m not familiar with changing my .bashrc or changing my path environment, and so I’m blogging about it here.

init and Python packaging (cf. Dave Forgac)

First, Dave Forgac’s 2015 PyOhio talk on Python Packaging from Init to Deploy gives a very lucid and understandable (grokkable?) introductory talk on the latest, best practices on (lazy) automating init and Python packaging.

Here’s Dave Forgac’s twitter page.

Requirements

Following Dave Forgac’s talk, when he actually gets into the code, after his very lucid and descriptive introduction, by minute 20:15, I pip installed the following:

pip install wheel
pip install twine
pip install tox

wheel creates new wheel format. twine for secure uploading to PyPI.
tox is a test runner.

Dealing with boilerplate: cookiecutter


pip install cookiecutter

Going to this page, I found suggested templates to “cookiecut” from in Available Cookiecutters. I wanted a minimal template, so I was pondering the use of these 2 templates, cookiecutter-pypackage-minimal: A minimal Python package template and cookiecutter-pipproject: Minimal package for pip-installable projects. I went with the latter for now, though the former looks intriguing as well.

Then I did this:

cookiecutter https://github.com/wdm0006/cookiecutter-pipproject.git

whence after filling out the names for my project, cookiecutter created locally in a subdirectory of my working directory a project template/boilerplate.

Working with cookiecutter-pipproject.git – “filling in the blanks”

Take a look at the files in this boilerplate template.

  • Dealing with installation requirements: This is completely handled by the requirements.txt file. Open it up in your favorite text editor (I use emacs) and type in each package your Python program requires (that you imported in with import). Press enter for each, so a new line is created. For example, here’s what I did for my project on Combustion chambers:

    cantera
    numpy
    scipy
    sympy
    BeautifulSoup
    requests

    This is my requirements.txt file.
  • LICENSE.md As Dave Forgac said, having a license is important, even if you don’t care about its ownership, because others can’t work or include it without any license at all. I went to choosealicense.com and copied the license I chose (MIT) and copied and pasted the text into LICENSE.md.

Miscellaneous links I found useful while googling around for “best practices python init init.py”:

Freeing up disk space on a Mac e.g. I’ve deleted everything off of Photos AND turned off iCloud Photo Library, but my disk space is still gone!

Today (20160318), I moved photos and videos I had on my iPhone 6 into Photos app by turning on, in Photos, Preferences-> iCloud, iCloud Photo Library box to check in on. Then my photos and vidoes were automatically downloaded from the iPhone 6, which had iCloud Photo Library turned on already. Then I was able to export all my photos and videos onto my external hard drive, because the Mac that I use, a MacBook Air, is 5 years old already and doesn’t have much free space available. I was able to grab 7 gbs of photos and videos off my iPhone 6, through iCloud, and onto my Photos app. And then I did File->Export->Export Unmodified Original and then saved appropriately in the New Folder I wanted on my external hard drive.

I saw that during the process, my disk space usage fluctuated from 7 gbs to 20 gbs (using up all my disk space, and thus Startup Disk gave me a warning).

After I grabbed the photos, and especially the 7 gb video off my iPhone 6 that I wanted (it was a lecture from school), I really wanted to turn off iCloud Photo Library on this Mac as it can’t really handle it with its cramped disk drive. I turned off the iCloud Photo Library option in Preferences of Photos app, deleted the photos, made an irrecoverable delete in the Recently Deleted folder (i.e. cleared out my Recently Deleted folder), and yet those 7 gb of disk space was not freed up from before.

I opened up Disk Utility (LaunchPad->Other->Disk Utility) and saw that “Other” was taking up alot more space than usual. I tried to look up on how to figure out what the breakdown of “Other” was, but it wasn’t very helpful cf. What Is Other Storage on Mac OS X and How to Remove It?; OS X: What is “other” space in About This Mac?.

Then I tried deleting my Photos Library (with Photos app closed). It appears that the so-called “Reference Library” can be moved around and doesn’t even have to be local: How to use external and referenced libraries with Photos for OS X. That didn’t help in recovering the majority of 7 gb. However, when I started up Photos on my Mac again, it prompted me to rebuild from my iPhoto Library the Photos Library (that was missing), and it did so, and that Photos Library took up 200 mb now, instead of 520 mb, which is a Photos Library where I deleted all my photos and videos!

To pinpoint where on my disk drive is all this “Other” files, this page with its commands proved useful:
OS X Yosemite: Increase disk space

and the commands, in Finder


Go -> Home
or press Shift-Command -H
File -> Find
, or press Command –F.

These commands allowed me to search on Finder depending on a huge list of attributes. I knew that the only modifications to my Mac was today, and so I searched by “Date Modified” and within 1 day. Taking a look at the file listing, I found what stuck out, what was “unusually” (subjective, you’ll have to do your own digging and judging) modified today, which was the file directory


/Users/myusername/Library/Containers/com.apple.cloudphotosd/Data/Library/Caches/CloudKit/Assets

And in that folder, I found my missing 10 gb, cached, and from Date Modified, Date Created, I knew that these were exactly the temporary files created during the process I described above, as I was doing nothing else with my computers and iOS devices.

I’m always wary of deleting over even modifying files in a Users Library, so I highly recommend caution in proceeding. Otherwise, I deleted the cached files (which had no file suffix endings and appears to be a computer-generated string of alphanumeric characters), after reading this post Safe to Delete Keynote Cache under /Library/Container/com.apple.iWork.Keynote/Data/Library/Caches

Other people had the same problem to with iCloud, iCloud Photo Library, and Photos app on a Mac:

Cloudd is consuming my harddisk constanly, http://forums.macrumors.com/threads/huge-40gb-cloudkit-cache.1791879/

Jupyter notebooks: Jupyter notebooks on wordpress; Jupyter notebooks on github

This link was useful in helping me to get a Jupyter notebook onto wordpress:
cf. Blogging in wordpress with ipython (jupyter) notebooks

Now, I am trying out Jupyter notebooks, to add to my workflow, as a way to quickly present and “put out there” my work coming out of Python and Sage Math. I looked into Jupyter notebooks because I noticed the sagemanifold guys, Michał Bejger and Eric Gourgoulhon use Jupyter notebooks. I decided on Jupyter notebooks because it was open-source, can export to html and pdf, and had Sage Math 7.1 support.

P.S. I can’t change the css anymore on wordpress.com ever since they made it a feature that you have to pay for, so I am skipping Step 3 of this link, Blogging in wordpress with ipython (jupyter) notebooks. And I am finding that wordpress.com isn’t rendering the tag. I am scrapping the idea of Jupyter notebooks on wordpress and going with github directly.

For example, my Jupyter notebook renders elegantly on github:

Presentations and Finitely Presented Groups

and github makes it easy to update off Terminal than wordpress.com.

C++, C++11/C++14

I add the important command of displaying matplotlib plots in jupyter notebooks:


%matplotlib inline

“Ten minutes to pandas” tutorial took me more than 10 minutes (about hour or more?). I like the interactivity of jupyter notebooks.

Links to help video, screen capture on Fedora Linux, and how to speed up video, for a time lapse video, on Fedora Linux

https://ask.fedoraproject.org/en/question/10272/how-can-i-capture-a-video-of-my-screen-in-fedora/

How to make a screencast using Vlc?

Select OPEN MEDIA

Select CAPTURE DEVICE tab

Change capture mode to DESKTOP

Change Options to 25 f/s

Choose CONVERT option instead of play option

In the Convert window add the destination file via the BROWSE button then name the file TEST.MPG (VLC does require the extension so the file becomes TEST.MPG rather than just TEST)

Change the PROFILE to VIDEO – H.264 + AAC (TS)

Click the setting button and change the VIDEO CODEC options to:

BITRATE 1200kb/s

FRAME RATE 25.00 fps

Click SAVE

Choose the DEINTERLACE option

Click START

http://superuser.com/questions/160432/speedup-a-video-on-linux?newreg=1e886589ea10442397e8373037048cea

ffmpeg -i input.mp4 -filter_complex "[0:v]setpts=0.5*PTS[v];[0:a]atempo=2[a]" -map "[v]" -map "[a]" -c:v libx264 -c:a aac output.mp4

Advertisements

One thought on “Computers

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s