Drive Crash 4: Now That's What I Call rsync: Difference between revisions

Jump to navigation Jump to search

Drive Crash 4: Now That's What I Call rsync (view source)

Revision as of 06:06, 17 June 2017

81 bytes added ,  17 June 2017
no edit summary
No edit summary
No edit summary
Line 1: Line 1:
At the end of the 2016/2017 academic year, during weeks 9and 10, [[Sam Willcocks]], [[Tom Lee]] and [[Matthew Stratford]] decided it would be a good idea to tear everything out of the AV and Computing racks (See [[The Great Tech Redo 2017]]). Unfortunately, some of the servers didn't like being turned off and moved around and decided to fail.
At the end of the 2016/2017 academic year, during weeks 9 and 10, [[Sam Willcocks]], [[Tom Lee]] and [[Matthew Stratford]] decided it would be a good idea to tear everything out of the AV and Computing racks (See [[The Great Tech Redo 2017]]). Unfortunately, some of the servers didn't like being turned off and moved around and decided to fail.


The first server to complain was backup, which complained of a degraded array. This was caused by a [[docs:Glossary | High Impedance Air Gap]] between the hard drive power supply and the drive.
The first server to complain was backup, which complained of a degraded array. This was caused by a [[docs:Glossary | High Impedance Air Gap]] between the hard drive power supply and the drive.
Line 30: Line 30:
Pizza was then ordered.
Pizza was then ordered.


The decision was made to prioritise current and paid productions from pending edits during the transfer; so these projects were synced to Backup first. Edwin then set about copying these projects from backup to his SSD so that YSTV definitely, absolutely, without a doubt had a copy.


All seemed well so the Tech/Computing teams went back to completing small jobs about the studio. Tim started working on chron jobs to automate backups, Tom started to assemble a media cache for the edit PCs as a way to combat drive failure, and Matt continued work on patching/routing various cables. All was fine, until Backup started reporting SMART errors.


Goddammit.


Attenborough disaster
Now there was a mad scramble to retrieve data from Backup onto Edwin's SSD.
  - Phone call to [[Sam Willcocks]] who is in sheffield after a job interview in London
  - Discussions about transferring data to [[Bruce]]
  - Decision is made to transfer data off of [[Attenborough]] to [[Backup]]
  - Pirates of the Caribbean soundtrack used to compliment the atmosphere (and keep spirits high)
  - Pizza ordered
  - Debian installed on another 500GB drive
  - Given the host name "TomScott" due to the new OS disk being a temporary bodge
  - ZFS pool mounted
  - The rsyncing begins
  - We rsync the most recent (and most important) productions to backup
  - [[Edwin Barnes]] starts transferring footage from Pending Edits backup to Edit 2 to edit
  - All seems well
  - [[Tim Bradgate]] starts working on chron jobs to auto backup
  - Tom starts working on a media cache PC/Ingest station
  - Matt continues working on patching/routing
  - Backup starts reporting SMART errors
  - Goddammit
  - Scramble to retrieve data from backup onto Edwin's SSD and one of YSTV's SSDs
  - Tom takes a break to have a shower while he has the opportunity
  - Shortly followed by Katherine and Hui-Ling
  - Meanwhile Matt and Tim pull Backup out of the rack
  - Tom returns and Tim goes for a shower
  - The 2TB drive in [[Obriain]] is sacrificed to Backup to rebuild the array
  - Matt and Tom take the opportunity during the rebuild to continue the ongoing attempt to tidy up the studio


Attendees
As there was not much else to do other than to wait for files to sync and pray that Backup lived long enough, Tom took this opportunity to go home, have a shower, and change - shortly followed by Katherine and Hui-Ling. Meanwhile, Matt and Tim pulled Backup out of the Computing rack. Upon Tom's return, Tim took his shift of showering and changing and Tom pulled the 2TB drive out of [[Obriain]] to be sacrificed to the great Backup RAID array. Matt and Tom take the opportunity during the rebuild to continue the ongoing attempt to tidy up the studio.
   - Katherine - Nervous and there
 
   - Hui-Ling - Cable Monkey
After Tim's return, Tim and Tom continue setting up the media cache while Tom continues his effort to write the wiki article for the 4<sup>th</sup> in the series of Drive Crashes while the crash unfolded around him.
   - Tom - Chief Bodger
 
   - Tim - Linux Wrangler
=Attendees=
   - Matt - Cat5 Patcher
   *Katherine - Nervous and there
   - Edwin - Stressed Out Editor
   *Hui-Ling - Cable Monkey
   - Sam - Remote Tech Support
   *Tom - Chief Bodger
   *Tim - Linux Wrangler
   *Matt - Cat5 Patcher
   *Edwin - Stressed Out Editor
   *Sam - Remote Tech Support
Retrieved from ‘https://wiki.ystv.co.uk/wiki/Special:MobileDiff/9673’

Navigation menu