Summary
This blog post explores the impact of disabling Oracle ASM Filter Driver filtering on space reclamation on a FlashArray system.
This article on Oracle ASM Filter Driver filtering originally appeared on Ron Ekins’s blog. It has been republished with the author’s credit and consent.
I’m a long-time fan of the Oracle ASM Filter Driver (AFD). One of its major benefits is AFD filtering.
AFD can ensure writes are only performed by trusted and registered Oracle processes, blocking potentially damaging writes that would have otherwise resulted in data corruption.
Unfortunately, a recent Linux kernel change has impacted the AFD and the latest recommendation from Oracle is to disable AFD filtering until a fix is available.
From Oracle Database 19.24, filtering is disabled by default, but for earlier versions, Oracle has provided Patch 35618858: Disable Filtering by Default for 5.15 and above Kernels.
Available Oracle Release Update (RU) Patches
- Oracle Clusterware 19 Release (19.23.0.0.240416ACFSRU)
- Oracle Clusterware 19 Release (19.24.0.0.0ACFSRU)
- Oracle Clusterware 19 Release (19.25.0.0.0ACFSRU)
In this blog post, I’ll explore the impact of disabling AFD filtering on space reclamation on all-flash storage arrays.
Kernel Version
I’ve recently upgraded my Oracle RAC servers to Oracle Linux 8.10 using kernel 5.15. You can check your kernel version using uname -r:
|
1 |
[oracle@z–rac1 ~]$ cat /etc/oracle–release |
|
1 |
Oracle Linux Server release 8.10 |
|
1 |
[oracle@z–rac1 ~]$ <strong>uname –r</strong> |
|
1 |
5.15.0–208.159.3.el8uek.x86_64 |
Oracle ASMFD State
We can use the asmcmd afd_state command to get the ASM Filter Driver status, for example:
|
1 |
[oracle@z–rac1 ~]$ asmcmd afd_state |
|
1 |
ASMCMD–9526: The AFD state is ‘LOADED’ and filtering is ‘ENABLED’ on host ‘z-rac1.uklab.purestorage.com’ |
In a RAC cluster, use the asmcmd afd_state –all option to see the status of all nodes. All nodes should have the same Filtering Status.
|
1 |
[oracle@z–rac1 ~]$ <strong>asmcmd afd_state —all</strong> |
|
1 |
node ‘z-rac1’: State of AFD: LOADED and ASMFD Filtering Status: ENABLED |
|
1 |
node ‘z-rac2’: State of AFD: LOADED and ASMFD Filtering Status: DISABLED |
Disable ASMFD Filtering
Disable filtering using the asmcmd afd_filter -d command:
|
1 |
[oracle@z–rac1 ~]$ <strong>asmcmd afd_filter –d</strong> |
|
1 |
[oracle@z–rac1 ~]$ <strong>asmcmd afd_state</strong> |
|
1 |
ASMCMD–9526: The AFD state is ‘LOADED’ and filtering is ‘DISABLED’ on host ‘z-rac1.uklab.purestorage.com’ |
Enable ASMFD Filtering
If required, you can re-enable filtering using the asmcmd afd_filter -e command, for example:
|
1 |
[oracle@z–rac1 ~]$ <strong>asmcmd afd_filter –e</strong> |
|
1 |
[oracle@z–rac1 ~]$ <strong>asmcmd afd_state</strong> |
|
1 |
ASMCMD–9526: The AFD state is ‘LOADED’ and filtering is ‘ENABLED’ on host ‘z-rac1.uklab.purestorage.com’ |
Storage Space Reclamation
Now, this raises a big question: If I disable the ASM Filter Driver, do I lose the ability to send SCSI UNMAP commands back to an all-flash storage array to release unused space?
OK, I have a two-node RAC cluster with ASMFD filtering DISABLED.
|
1 |
[oracle@z–rac1 ~]$ <strong>asmcmd afd_state —all</strong> |
|
1 |
node ‘z-rac1’: State of AFD: LOADED and ASMFD Filtering Status: DISABLED |
|
1 |
node ‘z-rac2’: State of AFD: LOADED and ASMFD Filtering Status: DISABLED |
Create Oracle Tablespace
To demonstrate this, let’s create a 4TB tablespace and wait for Oracle to initialise the datafile.
Let’s confirm there is no space being used by using my v$asm_disk query:
|
1 |
ASMFD Disks |
|
1 |
Disk Disk Disk Total Used Free |
|
1 |
Nbr Name Label Header Mount State Path GB GB GB |
|
1 |
—— ————— ————— ———– ———– ———– ————— ———— ———— ———— |
|
1 |
0 DATA08 DATA08 MEMBER CACHED NORMAL AFD:DATA08 12,288 0 12,288 |
|
1 |
1 DATA07 DATA07 MEMBER CACHED NORMAL AFD:DATA07 12,288 0 12,288 |
|
1 |
2 DATA05 DATA05 MEMBER CACHED NORMAL AFD:DATA05 12,288 0 12,288 |
|
1 |
3 DATA06 DATA06 MEMBER CACHED NORMAL AFD:DATA06 12,288 0 12,288 |
And now create the 4TB bigfile tablespace.
|
1 |
SQL> <strong>create BIGFILE tablespace test datafile ‘+AFDATA’ size 4T;</strong> |
|
1 |
Tablespace created. |
Using the v$asm_disk query, I can see my four ASMFD volumes have used in total 4TB as expected.
|
1 |
ASMFD Disks |
|
1 |
Disk Disk Disk Total Used Free |
|
1 |
Nbr Name Label Header Mount State Path GB GB GB |
|
1 |
—— ————— ————— ———– ———– ———– ————— ———— ———— ———— |
|
1 |
0 DATA08 DATA08 MEMBER CACHED NORMAL AFD:DATA08 12,288 1,024 11,264 |
|
1 |
1 DATA07 DATA07 MEMBER CACHED NORMAL AFD:DATA07 12,288 1,024 11,264 |
|
1 |
2 DATA05 DATA05 MEMBER CACHED NORMAL AFD:DATA05 12,288 1,024 11,264 |
|
1 |
3 DATA06 DATA06 MEMBER CACHED NORMAL AFD:DATA06 12,288 1,024 11,264 |
Using the asm command line utility asmcmd with the list attribute option, I see that thin provisioning is enabled on my AFDATA diskgroup:
|
1 |
[oracle@z–rac1 ~]$ <strong>asmcmd lsattr –G AFDATA –l | grep thin</strong> |
|
1 |
thin_provisioned TRUE |
Using the FlashArray™ CLI and performing the purevol list <volume name> –-space –-total command, we can confirm the space used by the tablespace in the AFD diskgroup. For example:
purevol list z-racpod::afd_data* –space –total
|
1 |
pureuser@z–x90–a> <strong>purevol list z–racpod::afd_data* —space —total</strong> |
|
1 |
Name Size Virtual Thin Provisioning Data Reduction Total Reduction Unique Snapshots Total |
|
1 |
z–racpod::afd_data05 12T 1.00T 92% >100 to 1 >100 to 1 4.95G 0.00 4.95G |
|
1 |
z–racpod::afd_data06 12T 1.00T 92% >100 to 1 >100 to 1 4.95G 0.00 4.95G |
|
1 |
z–racpod::afd_data07 12T 1.00T 92% >100 to 1 >100 to 1 4.95G 0.00 4.95G |
|
1 |
z–racpod::afd_data08 12T 1.00T 92% >100 to 1 >100 to 1 4.95G 0.00 4.95G |
|
1 |
(total) 48T 4.00T 92% >100 to 1 >100 to 1 19.79G 0.00 19.79G |
Drop Tablespace
Let’s now drop the Oracle tablespace using DROP TABLESPACE <tablespace_name> INCLUDING CONTENTS AND DATAFILES, and see the impact on the storage platform and Oracle database.
|
1 |
SQL> <strong>DROP TABLESPACE test INCLUDING CONTENTS AND DATAFILES;</strong> |
|
1 |
Tablespace dropped. |
If we re-run the v$asm_view query, Oracle is now reporting the 4TB test tablespace has been released.
|
1 |
ASMFD Disks |
|
1 |
Disk Disk Disk Total Used Free |
|
1 |
Nbr Name Label Header Mount State Path GB GB GB |
|
1 |
—— ————— ————— ———– ———– ———– ————— ———— ———— ———— |
|
1 |
0 DATA08 DATA08 MEMBER CACHED NORMAL AFD:DATA08 12,288 0 12,288 |
|
1 |
1 DATA07 DATA07 MEMBER CACHED NORMAL AFD:DATA07 12,288 0 12,288 |
|
1 |
2 DATA05 DATA05 MEMBER CACHED NORMAL AFD:DATA05 12,288 0 12,288 |
|
1 |
3 DATA06 DATA06 MEMBER CACHED NORMAL AFD:DATA06 12,288 0 12,288 |
Returning to the Pure Storage® FlashArray, we can see that we are still consuming space even though the Oracle tablespace has been deleted.
|
1 |
pureuser@z–x90–a> <strong>purevol list z–racpod::afd_data* —space —total</strong> |
|
1 |
Name Size Virtual Thin Provisioning Data Reduction Total Reduction Unique Snapshots Total |
|
1 |
z–racpod::afd_data05 12T 1.00T 92% >100 to 1 >100 to 1 4.95G 0.00 4.95G |
|
1 |
z–racpod::afd_data06 12T 1.00T 92% >100 to 1 >100 to 1 4.95G 0.00 4.95G |
|
1 |
z–racpod::afd_data07 12T 1.00T 92% >100 to 1 >100 to 1 4.95G 0.00 4.95G |
|
1 |
z–racpod::afd_data08 12T 1.00T 92% >100 to 1 >100 to 1 4.95G 0.00 4.95G |
|
1 |
(total) 48T 4.00T 92% >100 to 1 >100 to 1 19.79G 0.00 19.79G |
Oracle ASMFD Rebalance Compact Phase
When using Oracle ASM Filter Driver, we can manually trigger a SCSI UNMAP command to release space back to the storage array using the ALTER DISKGROUP <NAME> REBALANCE WITH BALANCE COMPACT.
I’ve included the optional WAIT option to only return once complete.
|
1 |
SQL> <strong>ALTER DISKGROUP AFDATA REBALANCE WITH BALANCE COMPACT WAIT;</strong> |
|
1 |
Diskgroup altered. |
If we now return to the FlashArray CLI and repeat the purevol list <volume name> –space –total command, we can see the impact of the ALTER DISKGROUP <name> REBALANCE WITH COMPACT.
We can see that even though AFD filtering has been disabled, ASMFD is still able to send the SCSI UNMAP request to the storage array to release the space.
|
1 |
pureuser@z–x90–a> <strong>purevol list z–racpod::afd_data* —space —total</strong> |
|
1 |
z–racpod::afd_data05 12T 1.00T 92% >100 to 1 >100 to 1 4.95G 0.00 4.95G |
|
1 |
z–racpod::afd_data06 12T 1.00T 92% >100 to 1 >100 to 1 4.95G 0.00 4.95G |
|
1 |
z–racpod::afd_data07 12T 1.00T 92% >100 to 1 >100 to 1 4.95G 0.00 4.95G |
|
1 |
z–racpod::afd_data08 12T 1.00T 92% >100 to 1 >100 to 1 4.95G 0.00 4.95G |
|
1 |
(total) 48T 4.00T 92% >100 to 1 >100 to 1 19.79G 0.00 19.79G |
|
1 |
pureuser@z–x90–a> <strong>purevol list z–racpod::afd_data* —space —total</strong> |
|
1 |
Name Size Virtual Thin Provisioning Data Reduction Total Reduction Unique Snapshots Total |
|
1 |
z–racpod::afd_data05 12T 92.00M 100% 6.9 to 1 >100 to 1 9.29M 0.00 9.29M |
|
1 |
z–racpod::afd_data06 12T 92.00M 100% 6.6 to 1 >100 to 1 10.47M 0.00 10.47M |
|
1 |
z–racpod::afd_data07 12T 92.00M 100% 6.7 to 1 >100 to 1 9.82M 0.00 9.82M |
|
1 |
z–racpod::afd_data08 12T 92.00M 100% 7.0 to 1 >100 to 1 8.63M 0.00 8.63M |
|
1 |
(total) 48T 368.00M 100% 6.8 to 1 >100 to 1 38.21M 0.00 38.21M |
Summary
In this blog post, I’ve confirmed that disabling ASMFD filtering has no impact on space reclamation.
The Oracle AFD development team is actively working on a new filtering and protection mechanism for the above-mentioned and newer kernels. When the new filtering solution becomes available, I will give it a try and share my findings.
If you want to learn more about using the ASM Filter Driver to reclaim storage space, check out this blog post.
Uncomplicate Oracle Data Storage
Take a Free Test Drive
Explore FlashArray in our virtual lab.
