After looking at the published solutions for FLARE-On 6, I realised that for challenge 11, vv_max, most people used a script to either reverse the AVX functions or to brute force it. My approach was different, I made use of memory breakpoints strategically placed at the address of the arrays to find out which AVX function modified the array and manually reversed each function to arrive at the 32 character input. Most of the AVX functions tend to involve 1 dynamic variable and 1 static variable which is not affected by the input.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75 | final_byte_arr = 70 70 B2 AC 01 D2 5E 61 0A A7 2A A8 00 00 00 00 08 1C 86 1A E8 45 C8 29 B2 F3 A1 1E 00 00 00 00
vpshufb_static = 02 01 00 06 05 04 0A 09 08 0E 0D 0C FF FF FF FF 02 01 00 06 05 04 0A 09 08 0E 0D 0C FF FF FF FF
vpmaddwd_static = 00 10 01 00 00 10 01 00 00 10 01 00 00 10 01 00 00 10 01 00 00 10 01 00 00 10 01 00 00 10 01 00
vpmaddubsw_static = 40 01 40 01 40 01 40 01 40 01 40 01 40 01 40 01 40 01 40 01 40 01 40 01 40 01 40 01 40 01 40 01
vpshufb_static2 = 00 10 13 04 BF BF B9 B9 00 00 00 00 00 00 00 00 00 10 13 04 BF BF B9 B9 00 00 00 00 00 00 00 00
vpand_static = 2F 2F 2F 2F 2F 2F 2F 2F 2F 2F 2F 2F 2F 2F 2F 2F 2F 2F 2F 2F 2F 2F 2F 2F 2F 2F 2F 2F 2F 2F 2F 2F
# The complete equation is as follows:
final_byte_arr = vpshufb(vpmaddwd(vpmaddubsw(vpaddb(input, intermediate_byte_arr), vpmaddubsw_static), vpmaddwd_static), vpshufb_static)
intermediate_byte_arr = vpshufb(vpand(vpsrld(input, 0x4), vpand_static), vpshufb_static2)
# Tracing backwards from final byte array
# Reverse vpshufb (Shift position of bytes based on index in mask)
IND 0 1 2 3 4 5 6 7 8 9 A B C D E F 0 1 2 3 4 5 6 7 8 9 A B C D E F
MAS 02 01 00 06 05 04 0A 09 08 0E 0D 0C FF FF FF FF 02 01 00 06 05 04 0A 09 08 0E 0D 0C FF FF FF FF
DES 70 70 B2 AC 01 D2 5E 61 0A A7 2A A8 00 00 00 00 08 1C 86 1A E8 45 C8 29 B2 F3 A1 1E 00 00 00 00
SRC B2 70 70 00 D2 01 AC 00 0A 61 5E 00 A8 2A A7 00 86 1C 08 00 45 E8 1A 00 B2 29 C8 00 1E A1 F3 00
# Reverse vpmaddwd (This function is rather hard to invert, I eyeballed the position of each nibble based on patterns)
IND 0 1 2 3 4 5 6 7 8 9 A B C D E F 0 1 2 3 4 5 6 7 8 9 A B C D E F
DES B2 70 70 00 D2 01 AC 00 0A 61 5E 00 A8 2A A7 00 86 1C 08 00 45 E8 1A 00 B2 29 C8 00 1E A1 F3 00
SRC 07 07 B2 00 C0 0A D2 01 E6 05 0A 01 72 0A A8 0A 81 00 86 0C AE 01 45 08 82 0C B2 09 3A 0F 1E 01
# Reverse vpmaddubsw (Just Math. 40x + 1y)
IND 0 1 2 3 4 5 6 7 8 9 A B C D E F 0 1 2 3 4 5 6 7 8 9 A B C D E F
MAS 40 01 40 01 40 01 40 01 40 01 40 01 40 01 40 01 40 01 40 01 40 01 40 01 40 01 40 01 40 01 40 01
DES 07 07 B2 00 C0 0A D2 01 E6 05 0A 01 72 0A A8 0A 81 00 86 0C AE 01 45 08 82 0C B2 09 3A 0F 1E 01
SRC 1C 07 02 32 2B 00 07 12 17 26 04 0A 29 32 2A 28 02 01 32 06 06 2E 21 05 32 02 26 32 3C 3A 04 1E
707 (40 x 1C) + (01 x 7)
0b2 (40 x 2) + (01 x 32)
ac0 (40 x 2b) + (01 x 0)
1d2 (40 x 7) + (01 x 12)
5e6 (40 x 17) + (01 x 26)
10a (40 x 4) + (01 x a)
a72 (40 x 29) + (01 x 32)
aa8 (40 x 2a) + (01 x 28)
081 (40 x 2) + (01 x 1)
c86 (40 x 32) + (01 x 6)
1ae (40 x 6) + (01 x 2e)
845 (40 x 21) + (01 x 5)
c82 (40 x 32) + (01 x 2)
9b2 (40 x 26) + (01 x 32)
f3a (40 x 3c) + (01 x 3a)
11e (40 x 4) + (01 x 1e)
# At this point, we have
vpaddb(input, intermediate_byte_arr) = 1C 07 02 32 2B 00 07 12 17 26 04 0A 29 32 2A 28 02 01 32 06 06 2E 21 05 32 02 26 32 3C 3A 04 1E
# It gets a bit tricky to reverse past this point because the 32 character input is eventually used in both inputs to the vpaddb function.
# However, for all remaining functions, a change in input at position X will only result in a change in output at position X.
# Hence, we can use a state table.
# vpaddb state table
# Note that the symbols and numbers have some overlap in output bytes.
A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F 10 11 12 13 14 15 16 17 18 19
a b c d e f g h i j k l m n o p q r s t u v w x y z
1A 1B 1C 1D 1E 1F 20 21 22 23 24 25 26 27 28 29 2A 2B 2C 2D 2E 2F 30 31 32 33
~ # $ % & ' ( ) * + , - . / 0 1 2 3 4 5 6 7 8 9 : ; < = > ?
37 36 37 38 39 3A 3B 3C 3D 3E 3F 40 41 3F 34 35 36 37 38 39 3A 3B 3C 3D 3E 3F 40 41 42 43
# The final answer is alphanumeric.
DST 1C 07 02 32 2B 00 07 12 17 26 04 0A 29 32 2A 28 02 01 32 06 06 2E 21 05 32 02 26 32 3C 3A 04 1E
SRC c H C y r A H S X m E K p y q o C B y G G u h F y C m y 8 6 E e
cHCyrAHSXmEKpyqoCByGGuhFyCmy86Ee
vv_max.exe FLARE2019 cHCyrAHSXmEKpyqoCByGGuhFyCmy86Ee
That is correct!
Flag: AVX2_VM_M4K3S_BASE64_C0MPL1C4T3D@flare-on.com
|
The graphic below demonstrates the eyeballing process for vpmaddwd to retrieve the reversed value. To be doubly sure that I did not make a mistake for any of the manually reversed byte arrays, I modified the value in memory and stepped through the function to confirm that the value returned is correct.
Bonus:
