HomeLatest ThreadsGreatest ThreadsForums & GroupsMy SubscriptionsMy Posts
DU Home » Latest Threads » Forums & Groups » Topics » Computers & Internet » Website, DB, & Software Developers (Group) » Would this work? Storage ...
Introducing Discussionist: A new forum by the creators of DU

Thu Feb 7, 2013, 09:03 PM

 

Would this work? Storage Device, SSD, 8PB for cheap?

Imagine this

We use commodity HW, and create SAN/NAS device that uses SD Chips

These can be very small, and not need a lot of cooling

Imagine instead of disks, you put in "wafers" that are cards with about 2-3TB of space, and each of these can be added to a storage unit/filesystem that could eventually hold 8PB in under 2U of rack space?

14 replies, 1409 views

Reply to this thread

Back to top Alert abuse

Always highlight: 10 newest replies | Replies posted after I mark a forum
Replies to this discussion thread
Arrow 14 replies Author Time Post
Reply Would this work? Storage Device, SSD, 8PB for cheap? (Original post)
Taverner Feb 2013 OP
Taverner Feb 2013 #1
TheMadMonk Feb 2013 #2
Taverner Feb 2013 #3
TheMadMonk Feb 2013 #4
Taverner Feb 2013 #5
Taverner Feb 2013 #8
Taverner Feb 2013 #10
Recursion Feb 2013 #6
Taverner Feb 2013 #9
Recursion Feb 2013 #11
Taverner Feb 2013 #12
Recursion Feb 2013 #13
Taverner Feb 2013 #14
Phillip McCleod Feb 2013 #7

Response to Taverner (Original post)

Thu Feb 7, 2013, 09:06 PM

1. That is a "wafer" is a sheet with several 128GB SD chips on it

 

Thoughts?

Reply to this post

Back to top Alert abuse Link here Permalink


Response to Taverner (Original post)

Thu Feb 7, 2013, 09:50 PM

2. Yes/No. Cost remains a huge factor.

 

Solid state comes in at roughly 100 times the cost per GB as spinning platters.

It's cheaper to pay for the space, the juice, and the juice necessary to run the AC at 20 bellow arctic.

Reply to this post

Back to top Alert abuse Link here Permalink


Response to TheMadMonk (Reply #2)

Thu Feb 7, 2013, 11:41 PM

3. Exactly, but say you didn't use SSDs but specifically cheap commodity HW?

 

that is, the SD mini chips that you put in your camera

RAID them all together for fault tolerance - and put them in sheets

Reply to this post

Back to top Alert abuse Link here Permalink


Response to Taverner (Reply #3)

Fri Feb 8, 2013, 03:54 AM

4. Because if you want petabytes of storage, you need...

 

...access speeds to match. Write times on these devices is often very slow.

It's doable, and being done. but it's not just a matter of grabbing 16000 x 32 GB thumb drive chips and cramming them into a box.

Nor will relying on ordinary RAID protocols, safeguard you against data loss. With SSDs, and a relatively fixed number of write cycles, it's quite probable that the backup device will fail within a very short timespan of the primary.

Reply to this post

Back to top Alert abuse Link here Permalink


Response to TheMadMonk (Reply #4)

Fri Feb 8, 2013, 12:08 PM

5. Ahhh makes sense

 

Thanks!

Reply to this post

Back to top Alert abuse Link here Permalink


Response to TheMadMonk (Reply #4)

Tue Feb 26, 2013, 02:59 PM

8. What about if you create multiple redunancies, not through RAID

 

But through something traditionally used for "Big Data" like Hadoop or Gluster?

Reply to this post

Back to top Alert abuse Link here Permalink


Response to TheMadMonk (Reply #4)

Tue Feb 26, 2013, 03:01 PM

10. BTW...why couldn't you use RAID?

 

Say, RAID 6 and multiple raid groups to make up a volume

Reply to this post

Back to top Alert abuse Link here Permalink


Response to Taverner (Original post)

Mon Feb 18, 2013, 08:21 PM

6. Finite writes make this a problem, and you'll need a logical filesystem that will be a PITA

You can only write to an SD a certain number of times before it goes kablooie. Now, you could have LEDs like RAID arrays do to show you which one to replace. But that gets to another point: what kind of redundancy do you want? RAID 6? 1+0? Something else? Write times and seek times are going to be problematic, and since the filesystem metadata and block map will have to be either in memory or on a faster filesystem, that's going to add a ton of complexity. That said, if you build one, I'll write the filesystem for it. That sounds fun.

Reply to this post

Back to top Alert abuse Link here Permalink


Response to Recursion (Reply #6)

Tue Feb 26, 2013, 03:00 PM

9. What about Hadoop for the FS?

 

You would have to do RAID 6, and not just that but several raid groups to make up one volume/FS

Reply to this post

Back to top Alert abuse Link here Permalink


Response to Taverner (Reply #9)

Tue Feb 26, 2013, 03:03 PM

11. Doesn't Hadoop rely on an underlying block FS?

I haven't looked at that stuff in a while.

Reply to this post

Back to top Alert abuse Link here Permalink


Response to Recursion (Reply #11)

Tue Feb 26, 2013, 03:06 PM

12. I don't know - but you could use Gluster for the FS

 

or really any good base block FS that has redundancy

Reply to this post

Back to top Alert abuse Link here Permalink


Response to Taverner (Reply #12)

Tue Feb 26, 2013, 03:11 PM

13. How about this: format each SIM card with FAT or something similarly fast and stupid

and then use each individual SIM card as a block in the overall FS. This could be cool.

Reply to this post

Back to top Alert abuse Link here Permalink


Response to Recursion (Reply #13)

Tue Feb 26, 2013, 03:16 PM

14. Hmmm I like that

 

Reply to this post

Back to top Alert abuse Link here Permalink


Response to Taverner (Original post)

Mon Feb 25, 2013, 09:23 PM

7. you're cool.

 

and you're all so damn smart! i do love DU

Reply to this post

Back to top Alert abuse Link here Permalink

Reply to this thread