We discuss to what extent information on ground-state properties of finite nuclei (energies and radii) can be used to obtain constraints on the symmetry energy in nuclear matter and its dependence on the density. The starting point is a generalized Weizsäcker formula for ground-state energies. In particular effects from the Wigner energy and shell structure on the symmetry energy are investigated. Strong correlations in the parameter space prevent a clear isolation of the surface contribution. Use of neutron skin-information improves the situation. We find that inclusion of the Wigner effect leads to a larger value of the symmetry energy of nuclear matter at the saturation point.