The result pictures basically look correct, although you might consider
an analog scope as a verification no aliasing is occuring due to scope
sampling rate if a low performance DSO.
I do see evidence you may have 60/120 Hz AM of the filter output, or
a ground loop with scope ground creating a ground moving with line
frequency. You might either float the scope ground, or better yet use
scope in diff mode, one channel on ground, one on filter out, and
invert one channel, scope mode set to add. That would get rid of CM
noise, in this case 60/120 Hz. Note scope in this mode has a spec for
CMRR itself, so look that up as it affects how much of the CM compo-
nents scope can reject. Remember to cal the probes as they also, if not
matched, affect CMRR in this mode.http://www.cbtricks.com/miscellaneous/tech_publications/scope/floating.pdf
The oversample clock is directly responsible for the filter pole/zero
locations, hence response, the wizard handles this for you. The res-
ponse of the filter controls the harmonic content in the filter output.
Clearly you want to reject harmonics as much as possible. So make sure
filter peak response occurs at freq of interest, the FSK value, and that
its BW/Q appropriate for harmonic rejection you are trying to achieve.
Also be carefull signal path gain thru any amp and filter do not saturate
the filter output swing. In this case G << 1 as the output buffer does not
swing rail to rail, if you overdrive any element in signal path that produces
Also pay attention to data sheet graphs for peak voltage vs parameters, like