Fox at the top of Leath Walk Edinburgh.
Kind: Articles
traditional long form content: a post with an explicit title and body
Loch Ard Reflections
Flickr 2014 and DIY pummelvision
A few years back I used pummelvision to make a video of all of my flickr photos. Pummelvision was an online service where you pointed it to your flickr stream and it built a video for you and posted to vimeo. It could also take input from tumblr and facebook.
I though it might be interesting to make a similar video for my photos this year. However going to look for pummelvision drew a blank, the company had closed. I then though It might be interesting to try to create a similar video. From my memory and looking at my old video, pummelvision made a video with no transitions and very fast. As far as I remember it just used one tune. I downloaded my old video from vimeo and extracted the audio file using QuickTime pro. I took a guess that the frame rate was about 6 photos per second.
Grabbing the images
I guess there a few ways to grab all your photos from flickr, but this is how I did it. If I was doing it frequently I’d look into automating it, but this was a once off, or once a year if I do it again.
Flickr’s api would allow you to do this, but it seemed a bit excessive to try and write a pile of code. The Flickr API has a section to test all of the command so I headed to: Flickr Api Explorer – flickr.photos.search. There I put my own user ID in, set the min_date_taken and max_date_taken, increased the per_page to 500 and added the large photo url to the extras field.
This produced an xml file will all the information:
I then extracted the 397 urls from the text. There will be many ways to do this, but I am experimenting with the Sublime Text application at the moment, it found & selected all of the https: occurrences and the with cmd-shift-right arrow expanded the selection to the quotes. One copy got all of the files!
Once I had a list of urls I edit those so that each line was:
curl "https://farm4.staticflickr.com/3897/14598292323_ae6462fa07_b.jpg" > image_183.jpg
With the numbers out the image sequential and padded to 3 characters, eg image_001.jpg, image 002.jpg etc. I also numbered then in reverse so the oldest photo would be first.
I saved this text as a file, dl.sh and moved into the terminal:
cd path/to/thefolder
chmod +x dl.sh
./dl.shThis code set the dl.sh file to executable and then ran it, the terminal filled with text and the folder with images. Curl is the command-line tool for downloading files.
Sizing images
Downloading the large size gave a folder full of images but some where landscape and some portrait, ie 1024 × 768 or 768 x1024 I need the images to all be the same size. So i used the sips utility to first resize them, sips --resampleHeight 768 *.jpg, then to pad the portrait ones: sips --padToHeightWidth 786 1024 *.jpg
Which gives me pictures like this for the portrait ones:
Making the movie
I discounted using iMove, moviemaker or the like as I wanted something quick (not necessary quick this time…) and that could be automated. I am also not sure in iMovie can show as fast as 6 per second. (Update, a quick look shows iMove can set speed to fractions of a second per frame.)
I though of a couple of ways to make the move, using Quicktime pro or ffmpeg. Quicktime pro proved the easiest option, opened the app and File -> Open Image sequence…, choose 6 frames per second, then all I had to do was save the movie.
Unfortunately Quicktime pro has been replace by Quicktime and it is a bit of a bother to get your old QT pro working if you had paid for a license. So I though I’d figure out ffmpeg too.
With great power comes great complexity
FFmpeg is A complete, cross-platform solution to record, convert and stream audio and video. It is a command line application and has a lot of variables. I can usually find out the right command with a bit of google. This one took quite a lot of google and failures. Most of these failures came from me trying to set a framerate, which lead to skipped frames. Eventually I dropped the idea of using the framerate options and got a very (too) fast video with this:
ffmpeg -f image2 -i IMG_%03d.jpg -c:v libx264 -pix_fmt yuv420p out.mp4
Note to self, in the -i, iput option IMG_%03d.jpg means all the images with 3 numerals, eg 001, 002… 375
I then slowed it down a little with this:
ffmpeg -i out.mp4 -filter:v "setpts=4.0*PTS" 2014-flickr-show.mp4
And added the audio:
ffmpeg -i 2014-flickr-show.mp4 -i pum.mp3 -map 0 -map 1 -codec copy -codec:a aac -strict experimental -b:a 192k -shortest 2014-flickr-show-audio.mp4
It took a fair bit of google to get the audio right too, the -codec:a option seems to sort things out.
Whys and Wherefores
As noted above, I could have done most of this with iMovie. But by using ffmpeg or QT pro, I’ve the opportunity to play, learn and possibly end up with an automated system. It would seem well within the realms of possibility to have a script that used the flickr api to download a bunch of images, perhaps for a year or with a tag and make a movie from them.
I’ve now figured out how to do most of this by piecing together the above fragments and finding out a bit about loops and renaming files, but I’ve no idea of how to create a bash script that will replace my hard coded tags, usernames ect with input, more to learn.
Once you have a lot of jpgs
You might as well do other things with them: 
iOS Workflow
I quite enjoy scripts and things that make my computing life a we bit easier. I’ve blogged a few times about AppleScript which I find very handy on my mac. On my iPhone I’ve never really found a way of automating things that stuck with me. I’ve downloaded and played with a few apps, but mostly they have felt a bit too convoluted for me.
I do regularly combine application to get a result, the so called app smashing, although I prefer the less destructive sounding playflow (I think I am the only person who does).
I’ve now found an application which looks like making this sort of thing on iOS a bit simpler: Workflow.
Workflow is more like Automator than AppleScript as it uses the same sort of block steps. You can combine any of the actions to create workflows. These steps or rather actions can deal with images, text, maps in all sorts of ways.
The think that makes this application stand out is that it has arrived hot on the heels of the iOS 8 improvements to inter application communication. You can set the application to the a Action Extension, this means it can be run from the share sheet in other applications. As you can set the input for a workflow to accept different things you can control the sheets where it will show.
In the screenshot below I’ve selected 2 photos and then hit the share button. When I click the Run Workflow button I can choose a workflow from the next screen(shown on the right) . In this case one choice is a simple workflow I made to downsize image an save it to the camera roll.
These workflows are made by dragging and dropping action blocks onto a workflow. Workflows can be set to be run from a icon added to your home screen, the Launch Center app or from share sheets in other apps. The latter can be set to accept different types of data and will then show up in the appropriate apps.
So far I’ve only made a few very simple workflows with two or three block, but there is potential to loop and have if-then type decision making.
There are over 150 actions you can use to build a workflow:
I’ve only scratched the surface of workflow over the last week or so, but it looks like it could make iOS more fun and effective.
A few links:
Friendly Monsters
Some recent links from my Pinboard. Mostly related to open resources.
- The Crusade for Curious Images | OpenGLAM
n December last year the British Library released over a million images on to Flickr Commons. …
… The Curious Images event held yesterday offered a whirlwind tour of the reuse of the images by artists, researchers and other institutions and of the challenges that tracking use and finding appropriate images continue to pose. - A fellow DS106 participant, Ron Leunissen from the Netherlands was among the winners GIF IT UP winners | OpenGLAM
GIF IT UP was an open competition to find the most excellent GIFs reusing openly licensed images and video from the collections searchable on the sites of the two digital libraries.
I think there is real potential for teaching about open licensing, copyright while having some fun and learning a bit about image editing through the creative use of openly licensed media.
- Monsters Are Real – Biodiversity Heritage Library a nice collection of open licence images. The Biodiversity Heritage Library works collaboratively to make biodiversity literature openly available to the world as part of a global biodiversity community.
- Lots more of this sort of thing at the OpenContentToolkit provide a gateway to contemporary and historical open digital media content from media archives and collections around the world. It is a space to explore, discuss and share examples of the use of open media at all school stages and at all levels of education.
- Not a open resource but
Lego Friends made me laugh and makes a point.
#glowblogs improvements: mobile
On of the real benefits to upgrading glow blogs to WordPress 4 will be its mobile interface. When WordPress 2.9.2 came out in 2010 we were just into the start of the mobile web and the term responsive web design had just been coined.
Now a large percentage of the population have devices in their pockets that are more than capable of posting content online.
. The WordPress dashboard now is responsive resizing and rearranging the tools to fit on my screen. Adding an image is simple and a gallery is easy enough.
WordPress now lets me select several images to insert in a post or create a gallery.
This post was started in the train, continued on the tube platform and finished on the couch all using my phone.
Personally I am not the greatest typest on any device. Many folk will be faster. Or I could start a post while mobile, capture images and save that as a draft for later.
I am looking forward to seeing how glow bloggers go mobile next year.
#glowblogs improvements: text handling
Since WordPress 3.9 the TinyMCE editor now automatically cleans out all but the semantic HTML from Word (or Rich text copied from elsewhere), meaning any fonts, styles, etc. You preserve headings, blockquotes, lists, links, bold, italic.
From: Peek in the SPLOT: TRU Writer
This means WordPress handles text pasted from word a lot better that the current WordPress 2.9.2 does
Even though, like Alan, I don't think writing everything in word is a great idea lots of folks do exactly that. This then should be useful for #glowblogs after the upgrade.
I'll be posting a few things here about the changes in glow blogs coming in January over the next few weeks.
Glow Blogs Phase 2
We have just announce the hope that phase 2 of the Glow Blogs project will go live sometime in January next year. This is a wee bit later than our original plans. Personally I am not too disappointed as I can see how hard the team across the Scottish Government and Code For The People (as was)1 are working.
Before I started this business, I could not really see what all the fuss was about, surely all we need to do is upgrade WordPress and all would be well. And that would be fine if it was one blog, dealing with, and sorting out any wee snags and glitches. But width >130000 blogs we need to try and make sure, as much as possible, that the blog owners don’t have much work to do. A major benefit of the newer version of WordPress should be making people’s lives easier it would be a bad start if they need to do a lot of work to keep their blogs looking the same.
Some Details
Here is an example or two of the work that is being carried out.
Anarchy
The Anarchy Media Player plugin was used to display video and audio on Glow Blogs. This plugin is no longer supported or updated. Much of its functionality has been improved on in core WordPress. WorpdPress now lets you upload or add a link to audio or video and choose to link to the media or embed it.
If we just turn off the plugin that would result in existing links to media, that are embedded, just being presented as links. If we leave it in folk may continue to use the features which may not survive any future upgrades. After a bit of though the developers have produced a slimmed down version of the plugin that will continue to change links into embeds. This new version will not however present buttons on the posting tool bar and encourage its continued use.
Hopefully this will smooth the upgrade experience for some people.
Resize
The old glow blogs had an upload file limitation of 8mb, this has been increased to 50mb which should help people to post small videos and reasonable length audio files. There will be extra cost in hosting the files and we want to balance this out a bit. In the old blogs quite a few folk ran out of space just by uploading un-edited image files. A plugin, resize at upload plus was added to the service in the hope that people would turn this on, images would be resized and very large sizes would be remove.
In the new setup this plugin has been updated and will be turned on for all blogs. The images will be resized down to 1200 pixels maximum width. This will ensure that images will still look good but trim some size of the rather large images that come out of digital cameras. This resize can be turned off, just in case someone want to have a photography blog and allow huge images to be stored.
These are just two of many changes that the team are working on. There is also the effect of the upgrade on the themes used on blogs, particularly the > 60 profile/e-portfolio themes. Each change needs a fair bit of discussion and work from the developers, each bit of functionality needs to be tested and all the other functionality need to be tested to see if the change has unforeseen consequences.
The User’s Experience
This is going to improve, WordPress has always been regarded as a good bit of software from an end user perspective. Improvement however means change. I am confident that these changes will make things a lot better, but habits will need to adapt. Hopefully we can explain any points of possible pain before the upgrade. We will be changing centuries in software terms, 2.9.2 was released in 2010 version 4 came out this year.
As usual I am more than happy to discuss any aspect of glow blogs, please get in touch if you need more information.
Footnotes:
1. Code for the people have joined the WordPress.com VIP Team at Automattic! They still are working on the Glow project.
Glow Blogs What Just Happened, What is Next?
Last Friday at the same time as the glow authentication changed, the new glow blogs service went live. I posted about this over on Glow Connect.
It was pretty exciting stuff, the developers were really working right up to the last minute and beyond to deliver the service. Even so we have gone live with a few know issues and have already discovered a few more.
At the start of the processes I certainly was not aware of all the complexities involved nor the scale of the job. Turns out it was a big complex job!
Luckily for me I ended up working with an amazing team, not only in the Scottish Government, but in the developers and suppliers. All of them worked long hours with very positive attitudes as I grumbled along. I am tempted to turn this post into a list of these characters and their qualities, but probably enough to say all of the blog team were essential to the process.
What Have We Got
Stray Puppy by p medved
Attribution-NonCommercial-ShareAlike License
At the end of this phase we have a working set of WordPress MU, one for each Local Authority, running the same version of the software as before and we have the known issues linked above.
The main improvement so far is around blog creation. There is no connection to SharePoint/old glow groups. This simplifies the process a great deal. This and other Main Changes are listed in the Glow Blog Help, some of these are not improvements but changes.
One more improvement, not listed, is that you can now upload files of up to 50mb to blogs. This should make it a lot easier to podcast or share small videos without using a third party site or service.
On Wednesday I popped back to North Lanarkshire to watch my colleague Ann McCabe set up a class of e-portfolios, this was much quicker than before, taking away at least half the steps. There is still plenty of room from improvement and I got a great idea to take back to Victoria Quay from the RM help desk who I visited in the afternoon.
Next Up
Given the above, if this was the end point in the process I’d be pretty disappointed. A lot of work for not much in the way of improvement. I am not disappointed due to two things, phase 2 and phase 3.
Phase 2 was looking quite simple, upgrade to a new version of WordPress. This will bring a host of benefits, better user experience especially on mobile being the main gain. More important, in the longer run, is that it gives us a much better base to develop on.
The other aspect of phase 2 will be to backfill in things that were dropped out of phase one or needs that were discovered in phase 1.
It looks like phase 2 will take a bit more work than I expected, but this will start straight off. Already some of the first problems to be discovered has been solved and the developer team are just waiting to decide when to deploy the code. Another potential ongoing problem with server load is now beginning to be understood and the team are working on finding the best solution. The team are keen that the server gets a chance to bed in and are suitably cautious about changing things on the live system, best practises for ongoing change and development are being put in place.
Bright Future
After we get to phase 3 of the project things might speed up a bit. We will be using WordPress 4 which will allow a lot of nice things to happen.
The one I am most excited about is giving a more flexible service. In the old glow blogs it was a constant frustration for myself and many others that our theme and plugin requests were never answered. I am not entirely sure of all the reasons for this, but having peeped behind the curtain I presume some of this was to do with testing.
Watching the new blogs service develop gave me a bit of a shock in the amount of time and effort it needs to deliver a service of this scale. Like many folk who publish stuff on the web I frequently make changes without much of a care and worry. The Technical Architects and developers for glow take a somewhat different view. There first concern is the preservation of users data and stability of the service and given they are taking care of over 100,000 blogs…
The glow blogs system now consists of 4 main servers: integration (where new code is added after code review), explore (for testing), pre-production (more testing) and live. With the older version of WordPress we are using a lot of the development and testing is manual, the testers here and volunteers going through lists of test to test the functionality of the blogs. In addition there is security, load and a many more tests.
Going forward the process should be automated, the newer version of WordPress can have a deal of automatic testing, code going onto the integrate server would be pushed through the different servers being automatically tested on the way, this gives us the possibility of a much more agile service.
On the Way
On Friday last week there was a fair amount of cheering and happy faces around the glow office, since then feedback has been mainly positive. I am not really ready to celebrate yet, there still is a lot to do before we reach the point learners and teachers in Scotland have a world class blogging platform. There so many possibilities out there for doing all sorts of things with WordPress. We would, of course be really interested to hear of any ideas of what you would like from Glow blogs.
Glow Blog Migration Dates
I posted this this afternoon:
The Glow Blogs migration export will start on 19th September 2014
Any new posts or content added to blogs after that date will not be migrated to the new service.
The new service should be up and running by Oct 3rd 2014
The data from the current blog servers will be exported on the 19th of September and migrated to the new service ready for the go live date.
We’re making every effort to achieve the deadline of the new service for 3rd October. If anything changes, we will get in touch immediately.
This is not technically a content freeze as users will be able to add to their blog, rather it should be considered as a procedural content freeze.
We hope to be able to add a message to explain the situation to every blog dashboard but in case that is not technically possible we need as much help as we can get in spreading the word.
from: Glow Blogs Migration News | Glow Connect, Glow Connect is the information portal for Glow – a space for providing updates on the development and enhancement of the service and for sharing how teachers are using Glow.
Joshi by Juan Coloma Attribution-NonCommercial-ShareAlike License
I am sorry that the warning time for the content freeze is only just over two weeks, but it is only now we can give an estimate, timing is tight, it might even slip a wee bit, but we though it best to try and give as much notice as we can.
Back in June we proposed a content freeze over a short period over the summer and possibly another in September or October, as it turned out the summer freeze did not happen as it would not have given us any advantage, we could do a text export without a freeze.
We (especially the coders, technical and test members of the team) are very much working flat out to get the blogs migration in on time, the date is the current best estimate of when we will be ready to export the data from the RM servers and import it into the new one.
Earlier we hoped that the freeze would be a bit shorter that we are now estimating, but it has become apparent that it will take a bit longer. Between the 19th and go live several things need to happen:
- The database and web server files (images and uploads) need to be encrypted and copied to a secure disk. Before encryption a sort of fingerprint of the files is taken, this will let us know if the files we put on the new server are identical. The size of the database and files meant this will take a while.
- The disk will be taken to the new hosting and copied onto the server.
- It is then unencrypted and the md5 fingerprint compared to the original.
- The files will be put in place and hooked up to WordPress, or rather 33 instances of WordPress one for each Local Authority plus a central one.
- Lots of testing. Testing of other bits of the process, the new servers has already started.
- There are several rounds of testing, of different types that I am just beginning to get my head round. This will insure we get the best possible service from the new blog. The final rounds of testing will involve users from across Scotland, first on a ‘test’ environment and then on the new server before it goes live.
- After everything looks good the new server gets the old blogs.glowscotland.org.uk domain and the blogs will be updatable again.
I’ve not numbered the points above because I have missed out many more steps. The project plan has been worked over repeatedly to make sure the quality of the result is as good as can be and, by doing various things in parallel, we cut time down to the minimum. Extra test engineers have been borrowed from other parts of glow and other members of the team are helping with testing.
After the new service goes live the project will not stop, the blogs will then be upgraded to a current version of WordPress and then the third phase of the project, to enhance the blogs for learning and teaching will start.