Touching or wiping the Kabah's cloth (Kiswah), its walls, or its corners — other than the Hajar Aswad and the Yemeni Corner — has no basis in the Sunnah and is considered an innovation by the majority of scholars. Nabi (shallallahu alaihi wa sallam) only performed istilam (ritual touching/kissing) of the Hajar Aswad, and he would touch the Yemeni Corner with his hand without kissing it. He deliberately did not touch the other two corners (the Iraqi and Shami corners) or the walls of the Kabah.
Sheikh Ibn Baz stated that many jamaah haji mistakenly rub the Kabah's cloth on their bodies, wipe their faces against the walls, or press their chests against the cloth believing it carries blessings (barakah). None of this was practiced by Nabi, and it has no basis in authentic evidence. The blessings of the Kabah are in performing Tawaf around it, praying near it, and making doa facing it — not in physical contact with its cloth or walls.
Sheikh Al-Albani emphasized that Nabi, who loved the Kabah more than anyone, only touched the Hajar Aswad and the Yemeni Corner. When Umar ibn al-Khattab kissed the Hajar Aswad, he said: 'I know that you are a stone and can neither benefit nor harm. Had I not seen the Messenger of Allah kissing you, I would not have kissed you.' This demonstrates that the touching is purely an act of following the Sunnah, not a belief that the stone itself holds power or blessings.