Merge series "spi: dw: Add full Baikal-T1 SPI Controllers support" from Serge Semin <Sergey.Semin@baikalelectronics.ru>:

Originally I intended to merge a dedicated Baikal-T1 System Boot SPI
Controller driver into the kernel and leave the DW APB SSI driver
untouched. But after a long discussion (see the link at the bottom of the
letter) Mark and Andy persuaded me to integrate what we developed there
into the DW APB SSI core driver to be useful for another controllers,
which may have got the same peculiarities/problems as ours:
- No IRQ.
- No DMA.
- No GPIO CS, so a native CS is utilized.
- small Tx/Rx FIFO depth.
- Automatic CS assertion/de-assertion.
- Slow system bus.
All of them have been fixed in the framework of this patchset in some
extent at least for the SPI memory operations. As I expected it wasn't
that easy and the integration took that many patches as you can see from
the subject. Though some of them are mere cleanups or weakly related with
the subject fixes, but we just couldn't leave the code as is at some
places since we were working with the DW APB SSI driver anyway. Here is
what we did to fix the original DW APB SSI driver, to make it less messy.

First two patches are just cleanups to simplify the DW APB SSI device
initialization a bit. We suggest to discard the IRQ threshold macro as
unused and use a ternary operator to initialize the set_cs callback
instead of assigning-and-updating it.

Then we've discovered that the n_bytes field of the driver private data is
used by the DW APB SSI IRQ handler, which requires it to be initialized
before the SMP memory barrier and to be visible from another CPUs. Speaking
about the SMP memory barrier. Having one right after the shared resources
initialization is enough and there is no point in using the spin-lock to
protect the Tx/Rx buffer pointers. The protection functionality is
redundant there by the driver design. (Though I have a doubt whether the
SMP memory barrier is also required there because the normal IO-methods
like readl/writel implies a full memory barrier. So any memory operations
performed before them are supposed to be seen by devices and another CPUs.
See the patch log for details of my concern.)

Thirdly we've found out that there is some confusion in the IRQs
masking/unmasking/clearing in the SPI-transfer procedure. Multiple interrupts
are unmasked on the SPI-transfer initialization, but just TXEI is only
masked back on completion. Similarly IRQ status isn't cleared on the
controller reset, which actually makes the reset being not full and errors
prone in the controller probe procedure.

Another very important optimization is using the IO-relaxed accessors in
the dw_read_io_reg()/dw_write_io_reg() methods. Since the Tx/Rx FIFO data
registers are the most frequently accessible controller resource, using
relaxed accessors there will significantly improve the data read/write
performance. At least on Baikal-T1 SoC such modification opens up a way to
have the DW APB SSI controller working with higher SPI bus speeds, than
without it.

Fifthly we've made an effort to cleanup the code using the SPI-device
private data - chip_data. We suggest to remove the chip type from there
since it isn't used and isn't implemented right anyway. Then instead of
having a bus speed, clock divider, transfer mode preserved there, and
recalculating the CR0 fields of the SPI-device-specific phase, polarity
and frame format each time the SPI transfer is requested, we can save it
in the chip_data instance. By doing so we'll make that structure finally
used as it was supposed to by design (see the spi-fsl-dspi.c, spi-pl022.c,
spi-pxa2xx.c drivers for examples).

Sixthly instead of having the SPI-transfer specific CR0-update callback,
we suggest to implement the DW APB SSI controller capabilities approach.
By doing so we can now inject the vendor-specific peculiarities in
different parts of the DW APB SSI core driver (which is required to
implement both SPI-transfers and the SPI memory operations). This will
also make the code less confusing like defining a callback in the core
driver, setting it up in the glue layer, then calling it from the core
driver again. Seeing the small capabilities implementation embedded
in-situ is more readable than tracking the callbacks assignments. This
will concern the CS-override, Keembay master setup, DW SSI-specific CR0
registers layout capabilities.

Seventhly since there are going to be two types of the transfers
implemented in the DW APB SSI core driver, we need a common method to set
the controller configuration like, Tx/Rx-mode, bus speed, data frame size
and number of data frames to read in case of the memory operations. So we
just detached the corresponding code from the SPI-transfer-one method and
made it to be a part of the new dw_spi_update_config() function, which is
former update_cr0(). Note that the new method will be also useful for the
glue drivers, which due to the hardware design need to create their own
memory operations (for instance, for the dirmap-operations provided in the
Baikal-T System Boot SPI controller driver).

Eighthly it is the data IO procedure and IRQ-based SPI-transfer
implementation refactoring. The former one will look much simpler if the
buffers initial pointers and the buffers length data utilized instead of
the Tx/Rx buffers start and end pointers. The later one currently lacks of
valid execution at the final stage of the SPI-transfer. So if there is no
data left to send, but there is still data which needs to be received, the
Tx FIFO Empty IRQ will constantly happen until all of the requested
inbound data is received. So we suggest to fix that by taking the Rx FIFO
Empty IRQ into account.

Ninthly it's potentially errors prone to enable the DW APB SSI interrupts
before enabling the chip. It specifically concerns a case if for some
reason the DW APB SSI IRQs handler is executed before the controller is
enabled. That will cause a part of the outbound data loss. So we suggest
to reverse the order.

Tenthly in order to be able to pre-initialize the Tx FIFO with data and
only the start the SPI memory operations we need to have any CS
de-activated. We'll fulfil that requirement by explicitly clearing the CS
on the SPI transfer completion and at the explicit controller reset.

Then seeing all the currently available and potentially being created
types of the SPI transfers need to perform the DW APB SSI controller
status register check and the errors handler procedure, we've created a
common method for all of them.

Eleventhly if before we've mostly had a series of fixups, cleanups and
refactorings, here we've finally come to the new functionality
implementation. It concerns the poll-based transfer (as Baikal-T1 System
Boot SPI controller lacks a dedicated IRQ lane connected) and the SPI
memory operations implementation. If the former feature is pretty much
straightforward (see the patch log for details), the later one is a bit
tricky. It's based on the EEPROM-read (write-then-read) and the Tx-only
modes of the DW APB SSI controller, which as performing the automatic data
read and write let's us to implement the faster IO procedure than using
the Tx-Rx-mode-based approach. Having the memory-operations implemented
that way is the best thing we can currently do to provide the errors-less
SPI transfers to SPI devices with native CS attached.

Note the approach utilized here to develop the SPI memory operations can
be also used to create the "automatic CS toggle problem"-free(ish) SPI
transfers (combine SPI-message transfers into two buffers, disable
interrupts, push-pull the combined data). But we don't provide a solution
in the framework of this patchset. It is a matter of a dedicated one,
which we currently don't intend to spend our time on.

Finally at the closure of the this patchset you'll find patches, which
provide the Baikal-T1-specific DW APB SSI controllers support. The SoC has
got three SPI controllers. Two of them are pretty much normal DW APB SSI
interfaces: with IRQ, DMA, FIFOs of 64 words depth, 4x CSs. But the third
one as being a part of the Baikal-T1 System Boot Controller has got a very
limited resources: no IRQ, no DMA, only a single native chip-select and
Tx/Rx FIFOs with just 8 words depth available. In order to provide a
transparent initial boot code execution the System Boot SPI Controller is
also utilized by an vendor-specific IP-block, which exposes an SPI flash
memory direct mapping interface. Please see the corresponding patch for
details.

Link: https://lore.kernel.org/linux-spi/20200508093621.31619-1-Sergey.Semin@baikalelectronics.ru/

[1] "LINUX KERNEL MEMORY BARRIERS", Documentation/memory-barriers.txt,
    Section "KERNEL I/O BARRIER EFFECTS"

Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Cc: Alexey Malahov <Alexey.Malahov@baikalelectronics.ru>
Cc: Ramil Zaripov <Ramil.Zaripov@baikalelectronics.ru>
Cc: Pavel Parkhomenko <Pavel.Parkhomenko@baikalelectronics.ru>
Cc: Andy Shevchenko <andy.shevchenko@gmail.com>
Cc: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Cc: Lars Povlsen <lars.povlsen@microchip.com>
Cc: wuxu.wu <wuxu.wu@huawei.com>
Cc: Feng Tang <feng.tang@intel.com>
Cc: Rob Herring <robh+dt@kernel.org>
Cc: linux-spi@vger.kernel.org
Cc: devicetree@vger.kernel.org
Cc: linux-kernel@vger.kernel.org

Serge Semin (30):
  spi: dw: Discard IRQ threshold macro
  spi: dw: Use ternary op to init set_cs callback
  spi: dw: Initialize n_bytes before the memory barrier
  Revert: spi: spi-dw: Add lock protect dw_spi rx/tx to prevent
    concurrent calls
  spi: dw: Clear IRQ status on DW SPI controller reset
  spi: dw: Disable all IRQs when controller is unused
  spi: dw: Use relaxed IO-methods to access FIFOs
  spi: dw: Discard DW SSI chip type storages
  spi: dw: Convert CS-override to DW SPI capabilities
  spi: dw: Add KeemBay Master capability
  spi: dw: Add DWC SSI capability
  spi: dw: Detach SPI device specific CR0 config method
  spi: dw: Update SPI bus speed in a config function
  spi: dw: Simplify the SPI bus speed config procedure
  spi: dw: Update Rx sample delay in the config function
  spi: dw: Add DW SPI controller config structure
  spi: dw: Refactor data IO procedure
  spi: dw: Refactor IRQ-based SPI transfer procedure
  spi: dw: Perform IRQ setup in a dedicated function
  spi: dw: Unmask IRQs after enabling the chip
  spi: dw: Discard chip enabling on DMA setup error
  spi: dw: De-assert chip-select on reset
  spi: dw: Explicitly de-assert CS on SPI transfer completion
  spi: dw: Move num-of retries parameter to the header file
  spi: dw: Add generic DW SSI status-check method
  spi: dw: Add memory operations support
  spi: dw: Introduce max mem-ops SPI bus frequency setting
  spi: dw: Add poll-based SPI transfers support
  dt-bindings: spi: dw: Add Baikal-T1 SPI Controllers
  spi: dw: Add Baikal-T1 SPI Controller glue driver

 .../bindings/spi/snps,dw-apb-ssi.yaml         |  33 +-
 drivers/spi/Kconfig                           |  29 +
 drivers/spi/Makefile                          |   1 +
 drivers/spi/spi-dw-bt1.c                      | 339 +++++++++
 drivers/spi/spi-dw-core.c                     | 642 ++++++++++++++----
 drivers/spi/spi-dw-dma.c                      |  16 +-
 drivers/spi/spi-dw-mmio.c                     |  36 +-
 drivers/spi/spi-dw.h                          |  85 ++-
 8 files changed, 960 insertions(+), 221 deletions(-)
 create mode 100644 drivers/spi/spi-dw-bt1.c

--
2.27.0

3 files changed, 42 insertions, 65 deletions

diff --git a/drivers/spi/spi-dw-core.c b/drivers/spi/spi-dw-core.c
index 55afdcee7d2b..d8e92f53e2bc 100644
--- a/drivers/spi/spi-dw-core.c
+++ b/drivers/spi/spi-dw-core.c
@@ -23,7 +23,6 @@
 /* Slave spi_dev related */
 struct chip_data {
 	u8 tmode;		/* TR/TO/RO/EEPROM */
-	u8 type;		/* SPI/SSP/MicroWire */
 
 	u16 clk_div;		/* baud rate divider */
 	u32 speed_hz;		/* baud rate */
@@ -105,7 +104,7 @@ void dw_spi_set_cs(struct spi_device *spi, bool enable)
 	 */
 	if (cs_high == enable)
 		dw_writel(dws, DW_SPI_SER, BIT(spi->chip_select));
-	else if (dws->cs_override)
+	else if (dws->caps & DW_SPI_CAP_CS_OVERRIDE)
 		dw_writel(dws, DW_SPI_SER, 0);
 }
 EXPORT_SYMBOL_GPL(dw_spi_set_cs);
@@ -142,11 +141,9 @@ static inline u32 rx_max(struct dw_spi *dws)
 
 static void dw_writer(struct dw_spi *dws)
 {
-	u32 max;
+	u32 max = tx_max(dws);
 	u16 txw = 0;
 
-	spin_lock(&dws->buf_lock);
-	max = tx_max(dws);
 	while (max--) {
 		/* Set the tx word if the transfer's original "tx" is not null */
 		if (dws->tx_end - dws->len) {
@@ -158,16 +155,13 @@ static void dw_writer(struct dw_spi *dws)
 		dw_write_io_reg(dws, DW_SPI_DR, txw);
 		dws->tx += dws->n_bytes;
 	}
-	spin_unlock(&dws->buf_lock);
 }
 
 static void dw_reader(struct dw_spi *dws)
 {
-	u32 max;
+	u32 max = rx_max(dws);
 	u16 rxw;
 
-	spin_lock(&dws->buf_lock);
-	max = rx_max(dws);
 	while (max--) {
 		rxw = dw_read_io_reg(dws, DW_SPI_DR);
 		/* Care rx only if the transfer's original "rx" is not null */
@@ -179,7 +173,6 @@ static void dw_reader(struct dw_spi *dws)
 		}
 		dws->rx += dws->n_bytes;
 	}
-	spin_unlock(&dws->buf_lock);
 }
 
 static void int_error_stop(struct dw_spi *dws, const char *msg)
@@ -204,7 +197,7 @@ static irqreturn_t interrupt_transfer(struct dw_spi *dws)
 
 	dw_reader(dws);
 	if (dws->rx_end == dws->rx) {
-		spi_mask_intr(dws, SPI_INT_TXEI);
+		spi_mask_intr(dws, 0xff);
 		spi_finalize_current_transfer(dws->master);
 		return IRQ_HANDLED;
 	}
@@ -228,7 +221,7 @@ static irqreturn_t dw_spi_irq(int irq, void *dev_id)
 		return IRQ_NONE;
 
 	if (!master->cur_msg) {
-		spi_mask_intr(dws, SPI_INT_TXEI);
+		spi_mask_intr(dws, 0xff);
 		return IRQ_HANDLED;
 	}
 
@@ -244,7 +237,7 @@ u32 dw_spi_update_cr0(struct spi_controller *master, struct spi_device *spi,
 
 	/* Default SPI mode is SCPOL = 0, SCPH = 0 */
 	cr0 = (transfer->bits_per_word - 1)
-		| (chip->type << SPI_FRF_OFFSET)
+		| (SSI_MOTO_SPI << SPI_FRF_OFFSET)
 		| ((((spi->mode & SPI_CPOL) ? 1 : 0) << SPI_SCOL_OFFSET) |
 		   (((spi->mode & SPI_CPHA) ? 1 : 0) << SPI_SCPH_OFFSET) |
 		   (((spi->mode & SPI_LOOP) ? 1 : 0) << SPI_SRL_OFFSET))
@@ -259,6 +252,7 @@ u32 dw_spi_update_cr0_v1_01a(struct spi_controller *master,
 			     struct spi_device *spi,
 			     struct spi_transfer *transfer)
 {
+	struct dw_spi *dws = spi_controller_get_devdata(master);
 	struct chip_data *chip = spi_get_ctldata(spi);
 	u32 cr0;
 
@@ -266,7 +260,7 @@ u32 dw_spi_update_cr0_v1_01a(struct spi_controller *master,
 	cr0 = (transfer->bits_per_word - 1);
 
 	/* CTRLR0[ 7: 6] Frame Format */
-	cr0 |= chip->type << DWC_SSI_CTRLR0_FRF_OFFSET;
+	cr0 |= SSI_MOTO_SPI << DWC_SSI_CTRLR0_FRF_OFFSET;
 
 	/*
 	 * SPI mode (SCPOL|SCPH)
@@ -282,6 +276,9 @@ u32 dw_spi_update_cr0_v1_01a(struct spi_controller *master,
 	/* CTRLR0[13] Shift Register Loop */
 	cr0 |= ((spi->mode & SPI_LOOP) ? 1 : 0) << DWC_SSI_CTRLR0_SRL_OFFSET;
 
+	if (dws->caps & DW_SPI_CAP_KEEMBAY_MST)
+		cr0 |= DWC_SSI_CTRLR0_KEEMBAY_MST;
+
 	return cr0;
 }
 EXPORT_SYMBOL_GPL(dw_spi_update_cr0_v1_01a);
@@ -291,20 +288,18 @@ static int dw_spi_transfer_one(struct spi_controller *master,
 {
 	struct dw_spi *dws = spi_controller_get_devdata(master);
 	struct chip_data *chip = spi_get_ctldata(spi);
-	unsigned long flags;
 	u8 imask = 0;
 	u16 txlevel = 0;
 	u32 cr0;
 	int ret;
 
 	dws->dma_mapped = 0;
-	spin_lock_irqsave(&dws->buf_lock, flags);
+	dws->n_bytes = DIV_ROUND_UP(transfer->bits_per_word, BITS_PER_BYTE);
 	dws->tx = (void *)transfer->tx_buf;
 	dws->tx_end = dws->tx + transfer->len;
 	dws->rx = transfer->rx_buf;
 	dws->rx_end = dws->rx + transfer->len;
 	dws->len = transfer->len;
-	spin_unlock_irqrestore(&dws->buf_lock, flags);
 
 	/* Ensure dw->rx and dw->rx_end are visible */
 	smp_mb();
@@ -323,7 +318,6 @@ static int dw_spi_transfer_one(struct spi_controller *master,
 	}
 
 	transfer->effective_speed_hz = dws->max_freq / chip->clk_div;
-	dws->n_bytes = DIV_ROUND_UP(transfer->bits_per_word, BITS_PER_BYTE);
 
 	cr0 = dws->update_cr0(master, spi, transfer);
 	dw_writel(dws, DW_SPI_CTRLR0, cr0);
@@ -445,7 +439,7 @@ static void spi_hw_init(struct device *dev, struct dw_spi *dws)
 	}
 
 	/* enable HW fixup for explicit CS deselect for Amazon's alpine chip */
-	if (dws->cs_override)
+	if (dws->caps & DW_SPI_CAP_CS_OVERRIDE)
 		dw_writel(dws, DW_SPI_CS_OVERRIDE, 0xF);
 }
 
@@ -462,12 +456,13 @@ int dw_spi_add_host(struct device *dev, struct dw_spi *dws)
 		return -ENOMEM;
 
 	dws->master = master;
-	dws->type = SSI_MOTO_SPI;
 	dws->dma_addr = (dma_addr_t)(dws->paddr + DW_SPI_DR);
-	spin_lock_init(&dws->buf_lock);
 
 	spi_controller_set_devdata(master, dws);
 
+	/* Basic HW init */
+	spi_hw_init(dev, dws);
+
 	ret = request_irq(dws->irq, dw_spi_irq, IRQF_SHARED, dev_name(dev),
 			  master);
 	if (ret < 0) {
@@ -498,9 +493,6 @@ int dw_spi_add_host(struct device *dev, struct dw_spi *dws)
 	device_property_read_u32(dev, "rx-sample-delay-ns",
 				 &dws->def_rx_sample_dly_ns);
 
-	/* Basic HW init */
-	spi_hw_init(dev, dws);
-
 	if (dws->dma_ops && dws->dma_ops->dma_init) {
 		ret = dws->dma_ops->dma_init(dev, dws);
 		if (ret) {
diff --git a/drivers/spi/spi-dw-mmio.c b/drivers/spi/spi-dw-mmio.c
index 18772c0c9220..c0d351fde782 100644
--- a/drivers/spi/spi-dw-mmio.c
+++ b/drivers/spi/spi-dw-mmio.c
@@ -48,13 +48,6 @@ struct dw_spi_mmio {
 #define SPARX5_FORCE_ENA			0xa4
 #define SPARX5_FORCE_VAL			0xa8
 
-/*
- * For Keem Bay, CTRLR0[31] is used to select controller mode.
- * 0: SSI is slave
- * 1: SSI is master
- */
-#define KEEMBAY_CTRLR0_SSIC_IS_MST		BIT(31)
-
 struct dw_spi_mscc {
 	struct regmap       *syscon;
 	void __iomem        *spi_mst; /* Not sparx5 */
@@ -204,7 +197,7 @@ static int dw_spi_mscc_sparx5_init(struct platform_device *pdev,
 static int dw_spi_alpine_init(struct platform_device *pdev,
 			      struct dw_spi_mmio *dwsmmio)
 {
-	dwsmmio->dws.cs_override = 1;
+	dwsmmio->dws.caps = DW_SPI_CAP_CS_OVERRIDE;
 
 	/* Register hook to configure CTRLR0 */
 	dwsmmio->dws.update_cr0 = dw_spi_update_cr0;
@@ -234,20 +227,13 @@ static int dw_spi_dwc_ssi_init(struct platform_device *pdev,
 	return 0;
 }
 
-static u32 dw_spi_update_cr0_keembay(struct spi_controller *master,
-				     struct spi_device *spi,
-				     struct spi_transfer *transfer)
-{
-	u32 cr0 = dw_spi_update_cr0_v1_01a(master, spi, transfer);
-
-	return cr0 | KEEMBAY_CTRLR0_SSIC_IS_MST;
-}
-
 static int dw_spi_keembay_init(struct platform_device *pdev,
 			       struct dw_spi_mmio *dwsmmio)
 {
+	dwsmmio->dws.caps = DW_SPI_CAP_KEEMBAY_MST;
+
 	/* Register hook to configure CTRLR0 */
-	dwsmmio->dws.update_cr0 = dw_spi_update_cr0_keembay;
+	dwsmmio->dws.update_cr0 = dw_spi_update_cr0_v1_01a;
 
 	return 0;
 }
diff --git a/drivers/spi/spi-dw.h b/drivers/spi/spi-dw.h
index f04075413dba..f88cf5cd5b05 100644
--- a/drivers/spi/spi-dw.h
+++ b/drivers/spi/spi-dw.h
@@ -2,6 +2,7 @@
 #ifndef DW_SPI_HEADER_H
 #define DW_SPI_HEADER_H
 
+#include <linux/bits.h>
 #include <linux/completion.h>
 #include <linux/debugfs.h>
 #include <linux/irqreturn.h>
@@ -70,6 +71,13 @@
 #define DWC_SSI_CTRLR0_FRF_OFFSET	6
 #define DWC_SSI_CTRLR0_DFS_OFFSET	0
 
+/*
+ * For Keem Bay, CTRLR0[31] is used to select controller mode.
+ * 0: SSI is slave
+ * 1: SSI is master
+ */
+#define DWC_SSI_CTRLR0_KEEMBAY_MST	BIT(31)
+
 /* Bit fields in SR, 7 bits */
 #define SR_MASK				0x7f		/* cover 7 bits */
 #define SR_BUSY				(1 << 0)
@@ -92,15 +100,16 @@
 #define SPI_DMA_RDMAE			(1 << 0)
 #define SPI_DMA_TDMAE			(1 << 1)
 
-/* TX RX interrupt level threshold, max can be 256 */
-#define SPI_INT_THRESHOLD		32
-
 enum dw_ssi_type {
 	SSI_MOTO_SPI = 0,
 	SSI_TI_SSP,
 	SSI_NS_MICROWIRE,
 };
 
+/* DW SPI capabilities */
+#define DW_SPI_CAP_CS_OVERRIDE		BIT(0)
+#define DW_SPI_CAP_KEEMBAY_MST		BIT(1)
+
 struct dw_spi;
 struct dw_spi_dma_ops {
 	int (*dma_init)(struct device *dev, struct dw_spi *dws);
@@ -114,7 +123,6 @@ struct dw_spi_dma_ops {
 
 struct dw_spi {
 	struct spi_controller	*master;
-	enum dw_ssi_type	type;
 
 	void __iomem		*regs;
 	unsigned long		paddr;
@@ -122,7 +130,8 @@ struct dw_spi {
 	u32			fifo_len;	/* depth of the FIFO buffer */
 	u32			max_freq;	/* max bus freq supported */
 
-	int			cs_override;
+	u32			caps;		/* DW SPI capabilities */
+
 	u32			reg_io_width;	/* DR I/O width in bytes */
 	u16			bus_num;
 	u16			num_cs;		/* supported slave numbers */
@@ -134,7 +143,6 @@ struct dw_spi {
 	size_t			len;
 	void			*tx;
 	void			*tx_end;
-	spinlock_t		buf_lock;
 	void			*rx;
 	void			*rx_end;
 	int			dma_mapped;
@@ -166,29 +174,19 @@ static inline u32 dw_readl(struct dw_spi *dws, u32 offset)
 	return __raw_readl(dws->regs + offset);
 }
 
-static inline u16 dw_readw(struct dw_spi *dws, u32 offset)
-{
-	return __raw_readw(dws->regs + offset);
-}
-
 static inline void dw_writel(struct dw_spi *dws, u32 offset, u32 val)
 {
 	__raw_writel(val, dws->regs + offset);
 }
 
-static inline void dw_writew(struct dw_spi *dws, u32 offset, u16 val)
-{
-	__raw_writew(val, dws->regs + offset);
-}
-
 static inline u32 dw_read_io_reg(struct dw_spi *dws, u32 offset)
 {
 	switch (dws->reg_io_width) {
 	case 2:
-		return dw_readw(dws, offset);
+		return readw_relaxed(dws->regs + offset);
 	case 4:
 	default:
-		return dw_readl(dws, offset);
+		return readl_relaxed(dws->regs + offset);
 	}
 }
 
@@ -196,11 +194,11 @@ static inline void dw_write_io_reg(struct dw_spi *dws, u32 offset, u32 val)
 {
 	switch (dws->reg_io_width) {
 	case 2:
-		dw_writew(dws, offset, val);
+		writew_relaxed(val, dws->regs + offset);
 		break;
 	case 4:
 	default:
-		dw_writel(dws, offset, val);
+		writel_relaxed(val, dws->regs + offset);
 		break;
 	}
 }
@@ -234,14 +232,15 @@ static inline void spi_umask_intr(struct dw_spi *dws, u32 mask)
 }
 
 /*
- * This does disable the SPI controller, interrupts, and re-enable the
- * controller back. Transmit and receive FIFO buffers are cleared when the
- * device is disabled.
+ * This disables the SPI controller, interrupts, clears the interrupts status,
+ * and re-enable the controller back. Transmit and receive FIFO buffers are
+ * cleared when the device is disabled.
  */
 static inline void spi_reset_chip(struct dw_spi *dws)
 {
 	spi_enable_chip(dws, 0);
 	spi_mask_intr(dws, 0xff);
+	dw_readl(dws, DW_SPI_ICR);
 	spi_enable_chip(dws, 1);
 }