On Baxter Q-operators and their Arithmetic Implications