Bayesian Priors

I know that it is difficult to actually create a prior that substantively influences a point estimate (except by design). But I always find it hard to justify a default prior. Is it merely that Ihave not spent enough time thinking about my data? Or am I missing some sort of trick to this process?