Four three-dimensional graphdiyne polymers, namely Tri-C18, Hex-C36, Tri-C54, and Orth-C72, are predicted with first principles calculations. These carbon allotropes are constituted with interlinked sp3-hybridized carbon pillars and sp2-hybridized graphene-like nanoribbons. Graphdiyne polymers are energetically more stable than C60 and graphdiyne, and show lower phase transition energy barriers from AA-graphdiyne than that of graphite–diamond transition. Except for the semimetallic Tri-C18, these polymers show narrow direct (or quasi-direct) band gap of 1.55–1.74 eV. The porous structure and the high in-plane Young’s modulus of graphdiyne sheet are inherited by these polymers. Meanwhile, a greatly enhanced Young’s modulus as high as ca. 1 TPa along the direction perpendicular to graphdiyne sheet and high hardness are developed in these systems.