1. 14 Apr, 2021 2 commits
    • Martin Storsjö's avatar
      arm64: filmgrain: Share the prologue of the fgy function · 54ad561d
      Martin Storsjö authored
      This is the same as what was done for the fguv function, to reduce
      the amount of space used for it (and also simplifying the calling
      code).
      
      This gives no significant slowdown for the case currently benchmarked
      by checkasm, while shrinking the code produced by film_grain.S by
      320 bytes.
      54ad561d
    • Martin Storsjö's avatar
      arm64: filmgrain: Add NEON implementation of the fguv function · 90bcb331
      Martin Storsjö authored
      Relative speedup over C code:
                                    Cortex A53    A72    A73   Apple M1
      fguv_32x32xn_8bpc_420_csfl0_neon:   4.51   2.87   3.88   6.51
      fguv_32x32xn_8bpc_420_csfl1_neon:   3.74   2.96   2.96   3.49
      fguv_32x32xn_8bpc_422_csfl0_neon:   4.49   3.18   4.07   5.00
      fguv_32x32xn_8bpc_422_csfl1_neon:   3.74   3.03   3.04   2.67
      fguv_32x32xn_8bpc_444_csfl0_neon:   6.68   4.24   5.66   5.02
      fguv_32x32xn_8bpc_444_csfl1_neon:   5.40   3.69   4.22   3.61
      90bcb331
  2. 15 Mar, 2021 1 commit
  3. 19 Feb, 2021 4 commits
  4. 12 Feb, 2021 1 commit
  5. 10 Feb, 2021 1 commit
    • Martin Storsjö's avatar
      arm64: itx16: Use usqadd to avoid separate clamping of negative values · 6f9f3391
      Martin Storsjö authored
      Before:                                Cortex A53     A72      A73
      inv_txfm_add_4x4_dct_dct_0_10bpc_neon:       40.7    23.0     24.0
      inv_txfm_add_4x4_dct_dct_1_10bpc_neon:      116.0    71.5     78.2
      inv_txfm_add_8x8_dct_dct_0_10bpc_neon:       85.7    50.7     53.8
      inv_txfm_add_8x8_dct_dct_1_10bpc_neon:      287.0   203.5    215.2
      inv_txfm_add_16x16_dct_dct_0_10bpc_neon:    255.7   129.1    140.4
      inv_txfm_add_16x16_dct_dct_1_10bpc_neon:   1401.4  1026.7   1039.2
      inv_txfm_add_16x16_dct_dct_2_10bpc_neon:   1913.2  1407.3   1479.6
      After:
      inv_txfm_add_4x4_dct_dct_0_10bpc_neon:       38.7    21.5     22.2
      inv_txfm_add_4x4_dct_dct_1_10bpc_neon:      116.0    71.3     77.2
      inv_txfm_add_8x8_dct_dct_0_10bpc_neon:       76.7    44.7     43.5
      inv_txfm_add_8x8_dct_dct_1_10bpc_neon:      278.0   203.0    203.9
      inv_txfm_add_16x16_dct_dct_0_10bpc_neon:    236.9   106.2    116.2
      inv_txfm_add_16x16_dct_dct_1_10bpc_neon:   1368.7   999.7   1008.4
      inv_txfm_add_16x16_dct_dct_2_10bpc_neon:   1880.5  1381.2   1459.4
      6f9f3391
  6. 09 Feb, 2021 2 commits
    • Martin Storsjö's avatar
      arm64: looprestoration: Rewrite the wiener functions · 2e73051c
      Martin Storsjö authored
      Make them operate in a more cache friendly manner, interleaving
      horizontal and vertical filtering (reducing the amount of stack
      used from 51 KB to 4 KB), similar to what was done for x86 in
      78d27b7d.
      
      This also adds separate 5tap versions of the filters and unrolls
      the vertical filter a bit more (which maybe could have been done
      without doing the rewrite).
      
      This does, however, increase the compiled code size by around
      3.5 KB.
      
      Before:                Cortex A53       A72       A73
      wiener_5tap_8bpc_neon:   136855.6   91446.2   87363.6
      wiener_7tap_8bpc_neon:   136861.6   91454.9   87374.5
      wiener_5tap_10bpc_neon:  167685.3  114720.3  116522.1
      wiener_5tap_12bpc_neon:  167677.5  114724.7  116511.9
      wiener_7tap_10bpc_neon:  167681.6  114738.5  116567.0
      wiener_7tap_12bpc_neon:  167673.8  114720.8  116515.4
      After:
      wiener_5tap_8bpc_neon:    87102.1   60460.6   66803.8
      wiener_7tap_8bpc_neon:   110831.7   78489.0   82015.9
      wiener_5tap_10bpc_neon:  109999.2   90259.0   89238.0
      wiener_5tap_12bpc_neon:  109978.3   90255.7   89220.7
      wiener_7tap_10bpc_neon:  137877.6  107578.5  103435.6
      wiener_7tap_12bpc_neon:  137868.8  107568.9  103390.4
      2e73051c
    • Kyle Siefring's avatar
      arm64: mc: Improve first tap for inorder cores · 4e869495
      Kyle Siefring authored
      Change order of multiply accumulates to allow inorder cores to forward
      the results.
      4e869495
  7. 06 Feb, 2021 1 commit
  8. 05 Feb, 2021 2 commits
    • Kyle Siefring's avatar
      arm64: warped motion: Various optimizations · a3b8157e
      Kyle Siefring authored
      - Reorder loads of filters to benifit in order cores.
      - Use full 128-bit vectors to transpose 8x8 bytes. zip1 is called in the
         first stage which will hurt performance on some older big cores.
      - Rework horz stage for 8 bit mode:
          * Use smull instead of mul
          * Replace existing narrow and long instructions
          * Replace mov after calling with right shift
      
      Before:            Cortex A55    A53     A72     A73
      warp_8x8_8bpc_neon:    1683.2  1860.6  1065.0  1102.6
      warp_8x8t_8bpc_neon:   1673.2  1846.4  1057.0  1098.4
      warp_8x8_16bpc_neon:   1870.7  2031.7  1147.3  1220.7
      warp_8x8t_16bpc_neon:  1848.0  2006.2  1121.6  1188.0
      After:
      warp_8x8_8bpc_neon:    1267.2  1446.2   807.0   871.5
      warp_8x8t_8bpc_neon:   1245.4  1422.0   810.2   868.4
      warp_8x8_16bpc_neon:   1769.8  1929.3  1132.0  1238.2
      warp_8x8t_16bpc_neon:  1747.3  1904.1  1101.5  1207.9
      
      Cortex-A55
      Before:
      warp_8x8_8bpc_neon:   1683.2
      warp_8x8t_8bpc_neon:  1673.2
      warp_8x8_16bpc_neon:  1870.7
      warp_8x8t_16bpc_neon: 1848.0
      After:
      warp_8x8_8bpc_neon:   1267.2
      warp_8x8t_8bpc_neon:  1245.4
      warp_8x8_16bpc_neon:  1769.8
      warp_8x8t_16bpc_neon: 1747.3
      a3b8157e
    • Kyle Siefring's avatar
      arm64: loopfilter: Avoid leaving 8-bits · 833382b3
      Kyle Siefring authored
      Avoid moving between 8 and 16-bit vectors where possible.
      833382b3
  9. 04 Feb, 2021 2 commits
  10. 28 Jan, 2021 4 commits
  11. 20 Jan, 2021 1 commit
    • Kyle Siefring's avatar
      arm64: cdef_dir: Preload rows to prevent stalling · 11cb2efa
      Kyle Siefring authored
      Before:            Cortex A53     A55     A72     A73
      cdef_dir_8bpc_neon:     400.0   391.2   269.7   282.9
      cdef_dir_16bpc_neon:    417.7   413.0   303.8   313.6
      
      After: Cortex A55
      cdef_dir_8bpc_neon:     369.0   360.2   248.4   273.4
      cdef_dir_16bpc_neon:    388.7   384.0   272.2   290.7
      11cb2efa
  12. 07 Jan, 2021 4 commits
  13. 16 Dec, 2020 1 commit
  14. 12 Dec, 2020 3 commits
  15. 01 Dec, 2020 3 commits
    • Martin Storsjö's avatar
      arm: looprestoration16: Fix comments referring to pixels as bytes · c58e9d57
      Martin Storsjö authored
      A number of other similar comments were updated to say pixels when
      the 16 bpc code was written originally, but these were missed.
      c58e9d57
    • Martin Storsjö's avatar
      arm64: looprestoration: Add a missed parameter in a comment · 25877c3b
      Martin Storsjö authored
      Make it consistent with the weighted1 function.
      25877c3b
    • Martin Storsjö's avatar
      arm64: looprestoration16: Don't keep precalculated squares in box3/5_h · cbd4827f
      Martin Storsjö authored
      Instead of calculating squares of pixels once, and shifting and
      adding the precalculated squares, just do multiply-accumulate of
      the pixels that are shifted anyway for the non-squared sum. This
      results in more multiplications in total, but fewer instructions,
      and multiplications aren't that much more expensive than regular
      arithmetic operations anyway.
      
      On Cortex A53 and A72, this is a fairly substantial gain, on A73
      it's a very marginal gain.
      
      The runtimes for the box3/5_h functions themselves are reduced
      by around 16-20%, and the overall runtime for SGR is reduced
      by around 2-8%.
      
      Before:                   Cortex A53       A72       A73
      selfguided_3x3_10bpc_neon:  513086.5  385767.7  348774.3
      selfguided_5x5_10bpc_neon:  378108.6  291133.5  253251.4
      selfguided_mix_10bpc_neon:  876833.1  662801.0  586387.4
      
      After:                    Cortex A53       A72       A73
      selfguided_3x3_10bpc_neon:  502734.0  363754.5  343199.8
      selfguided_5x5_10bpc_neon:  361696.4  265848.2  249476.8
      selfguided_mix_10bpc_neon:  852683.8  615848.6  577615.0
      cbd4827f
  16. 20 Nov, 2020 1 commit
  17. 24 Sep, 2020 2 commits
  18. 03 Sep, 2020 3 commits
  19. 02 Jul, 2020 2 commits