Touching or wiping the Kabe's cloth (Kiswah), its walls, or its corners — other than the Hacer-ul Esved and the Yemeni Corner — has no basis in the Sunnah and is considered an innovation by the majority of scholars. Hz. Peygamber (sallallahu aleyhi ve sellem) only performed istilam (ritual touching/kissing) of the Hacer-ul Esved, and he would touch the Yemeni Corner with his hand without kissing it. He deliberately did not touch the other two corners (the Iraqi and Shami corners) or the walls of the Kabe.
Sheikh Ibn Baz stated that many hacilar mistakenly rub the Kabe's cloth on their bodies, wipe their faces against the walls, or press their chests against the cloth believing it carries blessings (barakah). None of this was practiced by Hz. Peygamber, and it has no basis in authentic evidence. The blessings of the Kabe are in performing Tavaf around it, praying near it, and making dua facing it — not in physical contact with its cloth or walls.
Sheikh Al-Albani emphasized that Hz. Peygamber, who loved the Kabe more than anyone, only touched the Hacer-ul Esved and the Yemeni Corner. When Umar ibn al-Khattab kissed the Hacer-ul Esved, he said: 'I know that you are a stone and can neither benefit nor harm. Had I not seen the Messenger of Allah kissing you, I would not have kissed you.' This demonstrates that the touching is purely an act of following the Sunnah, not a belief that the stone itself holds power or blessings.