Giving up... for now

Answered

Comments

34 comments

  • Avatar
    Michal | CR
    Dear Peter,
    thank you for your post. We value your opinion very much.

    First of all. We are very thankful for all contributions to the forum of users and of our support guys. Our goal is not be arrogant in this forum. Our goal is to help our users to solve their issues. We definitely do not think that our software is perfect. Therefore please accept our apologies if there was written something arrogant in this forum.

    We will try to find threads you are talking about and we will discuss them with responsible persons.

    Let me just note that we have officially released PUBLIC selling of our product in February this year. Our software is still in Beta version and our competitors are on the market much longer time than we are. We are young team, and we are young company. We are doing our best to solve all issues and deliver all tools our users need.

    "-Failures due to hardware?": I must say that I disagree with you in this point. Of course our software tries to use all possible computational resources. However it is not the reason why it is so fast. The reason are different algorithms. Our algorithms work almost always in linear time to the number of inputs. That is what makes us unique.

    "- Crashes/Errors" : yes you are right there are issues, bugs, etc. just like in any other software. Are you working with Maya or 3D's Max? Well, I was using them some time during fbx support implementation recently and I got many HARD crashes. Of course, it can't be an excuse for our crashes. We want to have our product as much stable as possible. All screenshots you sent are either fixed now or there are known workarounds / reasons. Moreover, we have many satisfied customers that are using RC on daily basis so they must have stable workflows.

    "- Inconsistent outputs.": it is very hard to react to this point because it is like when you come to a doctor and tell him that "I'm Ill, please give me a medicine". He will have to do many tests and he will have to give you many questions first. We have to do the same first. If you wish then we will be more than happy to investigate with you these problems.

    "Lack of documented workflows for basic tasks. ", " modelling and orthophoto generation from drone footage " : I completely agree with you in these two points. We must improve documentation, especially we must create new tutorials. We must improve workflows and tools for "modelling and orthophoto generation from drone footage". Especially GCP placements etc.

    The biggest obstacle in this direction is that it is very hard to find a reliable expert person that can help us to create high quality tutorials etc. Therefore, maybe, here is also right place to ask readers of this thread: If you want to help us with tutorials then we will be more than happy to support you with free licenses and maybe start a collaboration, just contact us.

    Thank you again for the post.
    0
    Comment actions Permalink
  • Avatar
    Steven Caron
    I just wanted to mention that after a few auto software updates this past week (don't know which days you guys released them) I have found builds to be more stable AND my once 'wimpy' hardware can complete a normal quality build with the standard amount of 'image downscale'.

    @michalj, you mentioned some of peter's crashes have been fixed. Were those fixed recently in this auto update I did?

    Steven
    0
    Comment actions Permalink
  • Avatar
    peterwatson
    michalj,

    Thank you for taking the time to respond to my feedback/ criticisms. I guess I write out of frustration- especially of late as I just can't seem to get many successful outputs (maybe 1 out of 5 don't fail on me) - this used to be better but I don't seem to be having much luck at all in the last couple of weeks. Thus am very interested with Steven's post above... have there been any (?silent) updates in last few days that I may have missed? Maybe it's a case of the latest build fixing things for some users and breaking for others eg. me :( ?? It's difficult to isolate to a particular machine and hardware also due to the licencing requirements- can't just quickly change installation to another machine if main one fails processing.

    I can appreciate what you are saying as far as beta software and bug testing etc. I have beta tested software (and hardware) in the surveying industry for many years and I fully understand where RC sits vs other similar applications. Probably the term arrogance is a bit too harsh- just get a hint of deflecting user's issues back on themselves which I don't think is helpful towards software development and bug fixing. In my experience a large group of beta testers using software in real world scenarios is far more likely to find all those odd little bugs and generally do a better job of breaking the software than the programmers can do. I know in my case there's a high likelihood the results I'm getting aren't ideal as I haven't tweaked the software settings to suit my imagery and desired outcomes- but this comes back to lack of documentation which is acknowledged to be an issue at present. I would be quite open to developing a pdf workflow for what I do- ie. DSM/DTM and orthophoto generation using GCPs (construction surveying use) but I have lost inclination to do this lately as RC keeps failing to process anything of reasonable quality!

    Am planning on building a higher spec workstation in the next few weeks so I will try again once I've got this up and running and see if that solves some of my problems.

    thanks
    pete
    0
    Comment actions Permalink
  • Avatar
    Steven Caron
    Hey Peter

    I clicked on the question mark on the top right of the UI, next to the 'RC', and it says I am running version 1.0.2.2158. How about you?

    Steven
    0
    Comment actions Permalink
  • Avatar
    peterwatson
    Steven,
    I'm on the same build.

    pete
    0
    Comment actions Permalink
  • Avatar
    Steven Caron
    Hmm, that is a bummer... Before these updates my graphics driver would crash when doing normal quality on some projects and most definitely crash on high quality. My last 5 builds have run without crashes on normal and high quality.
    0
    Comment actions Permalink
  • Avatar
    peterwatson
    Yeah, just tried another run and it failed on normal detail- it worked on preview quality however (though the output orthophoto is not useful).
    What hardware are you running? I am using an Alienware core i7-2760QM @ 2.4GHz, 32Gb ram, 2x Nvidia GTX460M, SSD + HDD. I realize running a laptop is not ideal as cooling is not as good as a desktop but I run it on a cooling pad with 3 fans whenever I run these projects and it has worked "reasonably" well previously. I might try a clean install of windows 10 as my licence needs re-activating in the next couple of days (currently running Win7).

    Not sure what is going on- is it something in the software that has changed? or is my data not adequate? Last 2 projects I've attempted have been ~400 photos from drone flights at ~120m AGL. Flying a bit higher than I usually do and not as much overlap as I am just trying to achieve a basic orthophoto- not super high resolution DEM etc. Maybe the lower quality (compared to other projects) of these images is causing RC to fail (photoscan still processes OK however- just a lot slower). The reported errors for my GCPs (have 8 GCPs spread over project area) are all quite OK as far as I'm concerned... mind you the project fails whether I'm using GCPs or not so I doubt this is an issue.
    0
    Comment actions Permalink
  • Avatar
    chris
    400 seems to be pretty low number to be having issues with.

    I'm getting issues too, but last couple I've done around 700 its went out no problem on high.

    I've got issue's in the 2000 - 5000+ range while trying to get high detail scenes out.

    some of my issues have been camera alignment, if you have bad alignment it needs a lot more ram.

    but I'm also running 128gb of ram.
    0
    Comment actions Permalink
  • Avatar
    peterwatson
    Another flight today- flew lower (100m AGL) increased sidelap to 50% and forward lap to 60%, same area but this time ~700 photos.
    Aligned images Ok, reconstruction failed a couple of minutes in on Normal detail.

    Seems I can't get a useful output out of this at present (other than preview quality- which is pretty much just a lot of blobs )
    0
    Comment actions Permalink
  • Avatar
    chris
    are you shooting straight down?

    I'm find shooting at down 45 degree's with phantom 3 doing an orbits works pretty well. rather than shooting in the classic grid pattern.

    I did one today that worked well, only around 250 photos though, got a high detail model in a hour or so.

    now I'm just trying to add another 10000 ground level photos. and then I'll have another 2000 of so heli shots from previous shoot to add after that. I'll see how i go with all of these, but I'll probably run into issues.
    0
    Comment actions Permalink
  • Avatar
    chris
    are you also flying everything at the same height?

    I would suggest you want at least a few different heights. though i think more different random heights is better.

    it may be that a combination of grid flying, and then few different obits at different heights will work well.

    I'm still experimenting with this. and i should do some more testing.

    but i think at the taking for photo stage, think about creating 3d model, and not an ortho map or mosaic. they just happen to be easy outputs once you have the model.
    0
    Comment actions Permalink
  • Avatar
    peterwatson
    Thanks Chris,

    I'll give that a try. What are you using to do your angled shots or are you flying manually? i have been using dronedeploy to fly areas and it works pretty well, but only allows for nadir shots... maybe I could use litchi app... it allows for time lapse shots at whatever angle you want the gimbal I think.

    I'll do a quick test combining my 120m flight with the 100m flight for starters.

    Also going to try downgrading to previous download of RC- if that can be done using same licence??
    0
    Comment actions Permalink
  • Avatar
    peterwatson
    I seem to recall reading on another forum some guys getting good results from adding angled shots (doing stockpile surveys or construction stuff)- a bit of a no-no for old-school aerial photogrammetry- but maybe the software is so smart these days it not only handles this but benefits from it??
    0
    Comment actions Permalink
  • Avatar
    chris
    I haven't been personally flying our drone so much. but its basically been put into orbit mode. and having timelapse function going. and then a number of manual shots as well. (the ones from the same spot aren't ideal)

    I do need to look into to some flight control software and try some grid flights once i get a chance. let me know if some recommendations. I'm thinking about running a couple of offset grids, face the camera 45 degree down. and then maybe running it a couple of times offset and facing different directions.

    I also do heli photography as well. which is all angled.

    I do think it has be thought about differently to the old way of doing it though. since we aren't trying to make just a nice aerial photo that stitches well. its making a true 3d model. that just happens to be easy to take a top down view once its done.

    I almost think about it as if you were taking the photos for a small object, then enlarge that to do a big scene.

    are you flying a quadcopter or a plane?

    if you have access to the site again, you could try a few orbits at 40m, 70, 150m (if its a large site maybe you need a few at each height that overlap). etc... just to help stitch all your existing photos together.

    also pay attention to how much ram is being used. if its getting a error for out of memory. easy fix is to have a pagefile large enough.
    0
    Comment actions Permalink
  • Avatar
    peterwatson
    Site is about 1.0km x 2.5km and I will be there for next 18 months. So plenty of time for tweaking and refining of processes (if only I can get something to work).
    For grid flights I have used pix4d capture app (but it only allows rectangular areas and crashes a lot). Dronedeploy app works much better- allows variable shapes (can upload shapefiles for areas to map). Both allow you to enter flying altitude and overlaps and the software then generates flight path and flies in autopilot capturing images as required. Straight down images only however.

    Combining of 120m and 100m flights didn't work- I get this cuda error- 30- unknown error- so it's not a memory issue by the sounds of it- a graphics card problem.
    0
    Comment actions Permalink
  • Avatar
    peterwatson
    Grrrrrrr... another gripe- cannot wind back software updates. Will not allow uninstall and re-install of older builds- . So effectively I am left with a software that I have paid for but is no longer functional.
    0
    Comment actions Permalink
  • Avatar
    Steven Caron
    peterwatson wrote:
    What hardware are you running? I am using an Alienware core i7-2760QM @ 2.4GHz, 32Gb ram, 2x Nvidia GTX460M, SSD + HDD. I realize running a laptop is not ideal as cooling is not as good as a desktop but I run it on a cooling pad with 3 fans whenever I run these projects and it has worked "reasonably" well previously.


    My machine is a dog compared to that! i7-2820 @ 2.3 Ghz, 24gb ram, Nvidia Quadro 1000M, pulling off my external drive from a USB 3.0 port. :oops:
    0
    Comment actions Permalink
  • Avatar
    peterwatson
    i7-2820 @ 2.3 Ghz, 24gb ram, Nvidia Quadro 1000M

    That's very interesting.. what OS are you running?
    I will have to wait another week to change from windows 7 to 10 as I had to reactivate when I tried to downgrade the builds. Not that it worked anyway :(
    0
    Comment actions Permalink
  • Avatar
    Wishgranter
    Hi peterwatson

    The 4xx series was really power hungry GPU series... that is why even with additional cooling its a problem to get them working properly...

    If you would fill out the SIGNATURE could say it sooner :-( ucp.php?i=profile&mode=signature

    one more option as one of last resorts, look for some local EXPERT that can CLEAN the inner part of your notebook as over the years there could be a LOT of dust on the fans inside of the notebook..
    0
    Comment actions Permalink
  • Avatar
    peterwatson
    Wishgranter- I cleaned the inside of laptop not too long ago when I upgraded ram so i don't think that is an issue.

    Out of interest I re-ran a little project that I had successfully processed a couple of months ago (so a previous build of RC).
    Only 105 images, small flight over a couple of stockpiles- fantastic output.

    Guess what... it failed this time! Same thing "Unknown error, CUDA error : 30 :unknown error"
    There has got to be something going on in the latest build that has changed and it doesn't like my hardware or OS?

    Can someone reset my activation period so I can try this on same machine but with windows 10?
    0
    Comment actions Permalink
  • Avatar
    chris
    are you running dual gpu in a laptop?

    that looks like a gpu over heating error.

    if you are running dual, I'd disable one.
    0
    Comment actions Permalink
  • Avatar
    peterwatson
    chris wrote:
    are you running dual gpu in a laptop?

    that looks like a gpu over heating error.

    if you are running dual, I'd disable one.


    Yes. but doesn't make sense why it worked previously with 2x GPU for this small project and now it doesn't?? I'll give it a go however.
    0
    Comment actions Permalink
  • Avatar
    Steven Caron
    Hey Peter,

    I am running Windows 7...
    0
    Comment actions Permalink
  • Avatar
    peterwatson
    tried both combinations- 1x GPU, 2xGPU no luck fails in both instances
    0
    Comment actions Permalink
  • Avatar
    Wishgranter
    Hi peterwatson
    We have NOT touched the reconstruction algorithms for over 2 years.. So it does not depend on the actual RC build

    What is your PRECISE notebook model ? I would like to check its cooling capabilities

    The thing is you need to understand that NVDIA/AMD have their GPUs segmented per use-case.
    The x80 x70 are the power kings
    x60 x50 are middle class
    x40 x20 x10 are a LOW-end class

    This classification is done on power ( TDP ) and performance and few more things.
    What is nowhere presented so much is that their COOLING requirements are set per this segmentation. And mostly you can find a specification of occasional gaming, or fast processing etc.
    And this means the x40 - x10 GPU series is developed for certain power AND workload. The x40 - x10 are for OCCASIONAL gaming or work, nothing that lasts for a LONG time, as the DESIGN of the GPUs is not capable of sustained, high duration use. So in short, the cooling design and so on is NOT designed so that they can work for a longer time (30+ mins ), as the power and cooling is NOT capable of sustainability over longer time.
    This can be clearly seen on the reported user base, as CUDA error 30 ( + few more ) is related ONLY to the low-end GPUs.
    We have NO reports on this issues on x70 x80 cars in desktop PCs - if they are properly cooled !!
    There have been few x70 x80 in notebooks but, when we have inspected the THERMAL tests, we have found out that they have PROBLEMS with a proper longer sustained workload, because the notebooks have NOT been designed for this sort of work, mostly SLIM or very low-weight versions.

    Take a look here: http://www.notebookcheck.net/Review-Ali ... 257.0.html
    Scroll down to the TEMPERATURE
    However, the reader should note that this an extreme scenario (full load for several hours) which will probably not occur in an everyday use.
    BUT keep in mind that you have a DUAL 460M cards there !!!! so it raises the temperature a lot...
    0
    Comment actions Permalink
  • Avatar
    peterwatson
    Wishgranter, thanks for the info.
    Laptop model is Alienware M18x R1. Might look at upgrading GPUs to something like a GTX 765

    Image

    or 2 of these...
    http://www.ebay.com.au/itm/Top-Nvidia-GTX-770M-3GB-GDDR5-MXM-3-0b-Video-Card-for-DELL-ALIENWARE-M17-M18X-R1-/252512168188

    What I find odd is that this machine has been working OK- yes it would fail on bigger projects or on higher detail but now fails almost every time- now I can't even reconstruct data that I had successfully processed previously!? Also seems odd that the GPU is overheating (if that is the issue) so quickly now (within a minute or 2 of starting the reconstruction)- it's not like it's running for 30mins + before it fails.
    0
    Comment actions Permalink
  • Avatar
    Wishgranter
    Hi peterwatson
    Yup, a better solution than the 460M GPU...
    0
    Comment actions Permalink
  • Avatar
    Steven Caron
    I also find it odd that projects were working before for Peter and now is no longer working and projects that weren't working for me are working now? I know you say the algorithm hasn't changed but what did change between then and now? Surely, some bug fixes? I know I let RC send crash reports for many of my crashes.

    Also, as a developer of CUDA based programs and a speaker at GTC, wouldn't you want to lobby NVidia to provide better reporting for this type of failure? I mean "unknown error" isn't really acceptable. If the driver restarted because of thermal fail safe, then it should report as such. The projects I have run recently are running at the SAME temp as they were when they were failing. I ran a project for 6 hrs a few nights ago with sustained temps, it didn't give up on me.

    I would say, provide Peter with an earlier build and have him test it. Sometimes seemingly unrelated things are indeed related.

    BTW, I am happy to have stability now, I am just the type of person that wants to know how and why something works :)
    0
    Comment actions Permalink
  • Avatar
    peterwatson
    Waited for licence to renew and loaded RC on other work laptop. It is a HP zbook i7-4800MQ @2.7GHz, 32GB ram Nvidia quadro K3100M GPU

    Loaded up a new project with 335 images- ran for a couple of hours then failed- out of memory. I can't seem to get anything to work with this program anymore.

    Image
    0
    Comment actions Permalink
  • Avatar
    Wishgranter
    Hi peterwatson
    Contact me at milos.lukac@capturingreality.com. We have an internal build and it seems it could be the problem solver for this.. so a perfect testing environment..
    0
    Comment actions Permalink

Please sign in to leave a comment.