We all knew it was coming, with the never ending amounts of data we have coming from here/the former AG/HiddenPalace/TCRF and all the other groups we're finally at a stage we cannot keep up with the data.
I've been looking into the next best option and feel we should maybe get our own server/NAS for data. Clearly we cannot have this online to the public as we have to respect developers/copyright still. But I'm thinking we maybe do something like betaarchive and have our own servers for the data.
How would users feel about this? I don't mind hosting it home for awhile but my uploads not th fastest so maybe colocated servers would be easier. It would remove the need for multiple file hosting accounts and relying on trust of third party services. We'd control it fully from hardware down to software.
I know other groups are looking into similar options and feel we're at that stage too.
A NAS would cost us around £800 along with about £400/14TB drive (it's like £100 less for 12TB oddly so cost wise I think 12TB is best.) The NAS I was looking at can take 120TB total which would give us years of growth too. if we did a raid setup we'd lose some space but I think around 40TB would cover most of what we have so far along with other bits.
Would anybody be interested in something like this? We'd have to setup some security etc, but it's at the stage there's too much data to manage on a single laptop. I'm also leaning at cold archives like tapes but this is pretty expensive too, more so than a NAS setup.
Please let me/us know your views below if this would be viable.
EDIT: So looking into this more we could get an 8 bay NAS for £600, this one also allows us to expand to 18 drives for an extra £800 while keeping it one system. HDD wise Toshiba enterprise HDD's at 14TB cost £20 more than the 12TB one's and these are around £380/14TB now. We would need least two to start so just to get started this project would need around £1360 with poor download speeds. (due to rubbish UK net ?)
EDIT2: For cold storage LTO-8 is £2500 for the drive alone and around £120/30TB tape. We'd need least 4 here and a cleaning tape too.
I've been looking into the next best option and feel we should maybe get our own server/NAS for data. Clearly we cannot have this online to the public as we have to respect developers/copyright still. But I'm thinking we maybe do something like betaarchive and have our own servers for the data.
How would users feel about this? I don't mind hosting it home for awhile but my uploads not th fastest so maybe colocated servers would be easier. It would remove the need for multiple file hosting accounts and relying on trust of third party services. We'd control it fully from hardware down to software.
I know other groups are looking into similar options and feel we're at that stage too.
A NAS would cost us around £800 along with about £400/14TB drive (it's like £100 less for 12TB oddly so cost wise I think 12TB is best.) The NAS I was looking at can take 120TB total which would give us years of growth too. if we did a raid setup we'd lose some space but I think around 40TB would cover most of what we have so far along with other bits.
Would anybody be interested in something like this? We'd have to setup some security etc, but it's at the stage there's too much data to manage on a single laptop. I'm also leaning at cold archives like tapes but this is pretty expensive too, more so than a NAS setup.
Please let me/us know your views below if this would be viable.
EDIT: So looking into this more we could get an 8 bay NAS for £600, this one also allows us to expand to 18 drives for an extra £800 while keeping it one system. HDD wise Toshiba enterprise HDD's at 14TB cost £20 more than the 12TB one's and these are around £380/14TB now. We would need least two to start so just to get started this project would need around £1360 with poor download speeds. (due to rubbish UK net ?)
EDIT2: For cold storage LTO-8 is £2500 for the drive alone and around £120/30TB tape. We'd need least 4 here and a cleaning tape too.