Based on a detailed symmetry analysis, we state the general rules to build up the effective low-energy field theory describing a system of electrons weakly interacting with the lattice degrees of freedom. The basic elements in our construction are what we call the ``memory tensors,'' which keep track of the microscopic discrete symmetries into the coarse-grained action. The present approach can be applied to lattice systems in arbitrary dimensions and in a systematic way to any desired order in derivatives. We apply the method to the honeycomb lattice and reobtain the by-now well-known effective action of Dirac fermions coupled to fictitious gauge fields. As a second example, we derive the effective action for electrons in the kagome lattice, where our approach allows us to obtain in a simple way the low-energy electron-phonon coupling terms.