Touching or wiping the Ka'bah's cloth (Kiswah), its walls, or its corners — other than the Black Stone and the Yemeni Corner — has no basis in the Sunnah and is considered an innovation by the majority of scholars. The Prophet (peace be upon him) only performed istilam (ritual touching/kissing) of the Black Stone, and he would touch the Yemeni Corner with his hand without kissing it. He deliberately did not touch the other two corners (the Iraqi and Shami corners) or the walls of the Ka'bah.
Sheikh Ibn Baz stated that many pilgrims mistakenly rub the Ka'bah's cloth on their bodies, wipe their faces against the walls, or press their chests against the cloth believing it carries blessings (barakah). None of this was practiced by the Prophet, and it has no basis in authentic evidence. The blessings of the Ka'bah are in performing Tawaf around it, praying near it, and making dua facing it — not in physical contact with its cloth or walls.
Sheikh Al-Albani emphasized that the Prophet, who loved the Ka'bah more than anyone, only touched the Black Stone and the Yemeni Corner. When Umar ibn al-Khattab kissed the Black Stone, he said: 'I know that you are a stone and can neither benefit nor harm. Had I not seen the Messenger of Allah kissing you, I would not have kissed you.' This demonstrates that the touching is purely an act of following the Sunnah, not a belief that the stone itself holds power or blessings.