Imaging Processing Storage Question

Discuss what equipment, AP Software, AP Apps you are using.
Post Reply
User avatar
Ed217 United States of America
Jupiter Ambassador
Articles: 0
Offline
Posts: 263
Joined: Mon Aug 12, 2019 6:21 pm
4
Location: Virginia, USA
Status:
Offline

TSS Photo of the Day

Imaging Processing Storage Question

#1

Post by Ed217 »


I’m doing more and more astro imaging. Currently I use a laptop with an external hard drive that is about 2TB. All the imaging data is stored there for each night. I then later bring that back and transfer the files to a much larger desktop RAID system.

While that works fine in process, the amount of storage is getting very large. Last night was fairly clear the entire time. My focus was Jupiter and Saturn early in the evening, and the Moon later when it rose. I was out about 6 hours in total and ended up with about 350GB of image data. Virtually all of that is raw video captures. That is before any image processing like PIPP, Registax, GIMP, etc.

Last night I captured about 25 video streams ranging in size from 2GB to about 14GB. I expect that I will end up with maybe 4-5 best streams that end up as stacked images. In the past I have saved everything, but that’s beginning to get large in total size.

I am curious as to what approaches others use in both how your imaging data is stored and how much you save. What do you keep and what do you discard? How do you choose?
Scopes: Celestron Nexstar 8SE, Celestron C6N, WO Z61, WO GT71, iOptron RC6 .
Mounts: Celestron Nexstar, iOptron Sky Guider Pro, iOptron HEM27 w/iPolar
Cameras: Nikon D780, Nikon Zfc ,ZWO ASI224MC, ZWO ASI120MM-S, ZWO ASI294MC Pro, ZWO ASI485MC, ZWO ASI174MM, ASI662MC, ASI294MM, ASI2600MC Pro iPhone 14.
Tools: Raspberry PI4 with Stellarmate OS, Intel NUC Mini PC, Beelink MiniPCs.
Accessories: Celestron focus motor, ZWO EAF focusers, F6.3 field reducer, assorted eyepieces, 11" video monitor, polar scopes, iOptron iPolar
Computers: Dell desktops, MacOS MacMini, ASUS laptop, Intel NUC, Beelink MiniPD, PI, assorted software packages, iPhones, iPads
My AstroBin Images: https://www.astrobin.com/users/EdDixonImages/
User avatar
Lowjiber
Orion Spur Ambassador
Articles: 2
Offline
Posts: 975
Joined: Tue Apr 30, 2019 12:30 pm
4
Location: Las Vegas, Nv, USA
Status:
Offline

TSS Awards Badges

TSS Photo of the Day

Re: Imaging Processing Storage Question

#2

Post by Lowjiber »


I operate from a 2TB (I think) external drive for imaging captures. However, once I've processed what I want, I delete everything. Of course, since most of my captures are solar and solar changes literally every minute, so keeping old capture data is useless.

Having said that, I just stumbled across an image capture that had been "laying around" in my drive since 2016. :lol:

Clear Skies & Stay Safe
John (Urban Astronomer) Apertura AD10 Dob; XLT 150 Dob; XLT 120EQ; Lunt Solar 60 PT/B1200; ES AR102; SW Pro 100ED; 2 SW Pro 80ED's; 90mm Eq; WO Z-61; SW 90mm Virtuso Mak; 2 Orion ST-80's; Quark-C; Cams: Polemaster, ASI120MM-S, ASI174MM & ASI174MM-C
User avatar
Star Dad United States of America
Orion Spur Ambassador
Articles: 0
Offline
Posts: 744
Joined: Sat Aug 10, 2019 8:05 pm
4
Location: Norwich CT
Status:
Offline

TSS Awards Badges

Re: Imaging Processing Storage Question

#3

Post by Star Dad »


I started out saving my raw images on DVD - but after 30 DVDs I decided that wasn't very useful. Rotation of the images - either the camera or differing times usually means only the central area of the images are useful. So I started saving just the final stacked image - maybe to use with another stacked image later on - and the final post processed image on DVD. Even at that I fill a DVD with about 20 images.
"To be good is not enough when you dream of being great"

Orion 203mm/f4.9/1000mm, converted TASCO 114mm/f9/1000mm to steam punk, Meade 114mm/f9/1000, Coronado PST, Orion EQ-G, Ioptron Mini-Tower and iEQ30, Canon 70D, ASI120MM,ASI294MC, Ioptron SkyHunter
User avatar
JayTee United States of America
Universal Ambassador
Articles: 2
Offline
Posts: 5619
Joined: Thu Apr 25, 2019 3:23 am
4
Location: Idaho, USA
Status:
Offline

TSS Awards Badges

TSS Photo of the Day

Re: Imaging Processing Storage Question

#4

Post by JayTee »


I have a very difficult time of letting things go, especially images and video streams from a night's work. What I've learned about myself is that I thoroughly enjoy capturing those images and video streams and not so much the processing. What I now do is after I have processed the object say Jupiter or Saturn, I then get rid of all the video streams from that date, if I need more I'll go back and capture more.

Cheers,
JT
∞ Primary Scopes: #1: Celestron CPC1100 #2: 8" f/7.5 Dob #3: CR150HD f/8 6" frac
∞ AP Scopes: #1: TPO 6" f/9 RC #2: ES 102 f/7 APO #3: ES 80mm f/6 APO
∞ G&G Scopes: #1: Meade 102mm f/7.8 #2: Bresser 102mm f/4.5
∞ Guide Scopes: 70 & 80mm fracs -- The El Cheapo Bros.
∞ Mounts: iOptron CEM70AG, SW EQ6, Celestron AVX, SLT & GT (Alt-Az), Meade DS2000
∞ Cameras: #1: ZWO ASI294MC Pro #2: 662MC #3: 120MC, Canon T3i, Orion SSAG, WYZE Cam3
∞ Binos: 10X50,11X70,15X70, 25X100
∞ EPs: ES 2": 21mm 100° & 30mm 82° Pentax XW: 7, 10, 14, & 20mm 70°

Searching the skies since 1966. "I never met a scope I didn't want to keep."

Image
User avatar
mbocca United States of America
Mars Ambassador
Articles: 0
Offline
Posts: 179
Joined: Tue Sep 03, 2019 4:13 pm
4
Location: Irwin, PA, USA
Status:
Offline

TSS Photo of the Day

Re: Imaging Processing Storage Question

#5

Post by mbocca »


This is a great discussion topic. Typically, I will throw away all of the processed information except for maybe the results of a few key steps. I don't get rid of the original data unless there is an obvious problem with the images (out of focus, poor tracking etc.).

My thought is if I learn something new with processing, I will have data to work with. I have gone back and reprocessed data a couple of different times after picking up new workflows. The raw data by itself is not too bad to store, but if you tack on the calibrated, the corrected, and the aligned images you are 4x the data.

What I struggle with is the PixInsight project files. I store those for now, but they are on my processing computer, not the hard drive for the images. That takes a little pressure off of the images hard disk, but at some point I will have to decide about those as well.

Mike
Scopes: Explore Scientific ED102 Triplet APO, Celestron Edge 1100HD, Orion ST80
Mounts: Celestron CGE Pro, SkyWatcher Star Adventurer
Cameras: ZWO ASI1600mm Pro, Nikon D7100, Orion SSAG, ZWO ASI120MC
Control Software: Sequence Generator Pro, PHD2, ASCOM, Stellarium
Post Processing: PixInsight, GIMP 2.0
Post Reply

Create an account or sign in to join the discussion

You need to be a member in order to post a reply

Create an account

Not a member? register to join our community
Members can start their own topics & subscribe to topics
It’s free and only takes a minute

Register

Sign in

Return to “AP equipment / AP Software & Apps”