The construction of efficient proton transport channels in ion-conductive membranes is crucial to proton exchange membrane fuel cells (PEMFCs). Herein, zwitterion-functionalized covalent organic framework (Z-COF) with both ammonium groups and sulfonic acid groups was synthesized and blended with Nafion to prepare Nafion/Z-COF composite proton exchange membranes. The polymer-like feature of the Z-COF imparted favorable interactions between Z-COF and Nafion, and thus promoted the dispersion of Z-COF and the reorganization of ion clusters. The incorporation of Z-COF enhanced water retention property and the sulfonic groups on Z-COF provided additional proton-transport sites within the membranes. As a result, the low-energy-barrier paths for proton transport were created. The composite membranes with 10 wt% of Z-COF exhibited the highest proton conductivity of 0.22 S cm−1 at 80 °C and 100% RH, which was 57.1% higher than that of recast Nafion membrane. The enhanced proton conductivity also afforded the composite membrane a 45.7% increase in maximum power density of single fuel cell at 80 °C and 50% RH. The effects of Z-COF loading on membrane morphology, polymer chain mobility, thermal stability, water uptake and dimensional stability were also investigated.